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PREFACE 


This  effort  was  conducted  by  Purdue  University  under  the  sponsorship 
of  the  Rome  Air  Development  Center  Post-Doctoral  Program.  This  report 
describes  work  continued  by  Purdue  University  because  of  its  educational  and 
research  value.  The  motivation  for  the  research  was  a task  completed  in 
1975  for  the  Defense  Communications  Agency  under  the  direction  of  Drs. 

F.J.  Ricci  and  D.  Schutzer. 

The  RADC  Post-Doctoral  Program  is  a cooperative  venture  between  RADC 
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Syracuse  University  (Department  of  Electrical  Engineering)^ Purdue  University 
(School  of  Electrical  Engineering^  Georgia  Institute  of  Technology  (School 
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(Department  of  Electrical  Engineering)  act  as  prime  contractor  schools  with 
other  schools  participating  via  sub-contracts  w’ith  the  prime  schools'.  The 
U.S.  Air  Force  Academy  (Department  of  Electrical  Engineering) , Air  Force 
Institute  of  Technology  (Department  of  Electrical  Engineering) , and  the 
Naval  Post  Graduate  School  (Department  of  Electrical  Engineering)  also 
participate  in  the  program. 

The  Post-Doctoral  Program  provides  an  opportunity  for  faculty  at 
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INTRODUCTION 

In  the  effort  to  obtain  more  potential  computer  computational 
capability,  more  effective  ways  of  utilizing  computer  systems  and 
resource  sharing  between  geographically  dislocated  computer  systems, 
significant  effort  has  been  spent  over  the  last  few  years  in  the 
analysis,  design,  and  Implementation  of  computer  networks. 

The  first  computer  networks  were  established  because  of  the 
need  to  distribute  computer  services  to  a large  number  of  remote 
users.  In  such  a network  (usually  referred  to  as  "centralized") 
there  is  one  single  computer  facility  connected  to  a large  number 
of  passive  terminals  by  means  of  a communications  network.  [ROBE  67] 
suggested  that  existing  computer  systems  could  be  better  utilized  when 
interconnected  in  a network  so  that  their  resources  could  be  shared. 

Among  such  resources  we  mention  computer  power  (for  load  sharing); 
specialized  hardware;  specialized  software;  data  banks.  This  second 
type  of  network  differs  from  the  first  one  in  that  the  computer 
resources  are  distributed  among  the  nodes,  rather  than  accumulated 
in  the  central  node.  This  fact  also  suggests  better  survivability 
of  the  network.  This  configuration  is  hence  referred  to  as  a 
"distributed  computer  network." 

A very  important  component  of  the  computer  network  is  the  commun- 
ication network  which  indues  all  the  hardware  and  software  (modulators, 
demodulators,  channels,  concentrators,  interface  message  processors,  etc.) 
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specifically  dedicated  to  the  transfer  of  data  from  node  to  node.  In 
some  systems,  the  work  of  data  transmission  (message  block  assembly, 
storing,  routing)  is  shared  between  computers  and  communlcat ion  net- 
works; in  some  others,  it  is  completely  assigned  to  the  communication 
network. 

Many  different  communication  schemes  can  be  Implemented:  dial-up 
lines,  dedicated  lines,  circuit  switching,  store-and-forward  (also 
called  message  switching),  radio  broadcasting,  satellite  communications, 
and  loop  systems.  Of  course,  there  are  also  hybrid  systems  that  Include 
more  than  one  scheme  at  a time.  The  selection  of  the  best  scheme  for  a 
given  application  is  a very  difficult  problem  and  depends  on  many 
factors  (degree  of  decentralization  of  the  network,  traffic  character- 
istics, delay  requirements,  hardware  costs,  e:  ).  A systematic 
approach  to  the  general  problem  would  be  very  much  technology  and 
application  dependent.  The  selection  of  the  appropriate  type  of 
topoiogicai  structure  should  also  depend  on  the  application. 

The  co.omuni cat  Ions  network  considered  here  Is  of  the  store-and- 
forward  type.  In  such  a network  a message  Is  stored  at  each  node 
before  being  transmitted  to  the  next  node  in  the  path;  If  a link  is 
busy,  then  a message  can  be  held  in  storage  until  the  link  is  free. 

Within  the  network,  traffic  flow  is  in  terms  of  messages  in  their 
entirety  as  far  as  the  message  length  is  concerned.  The  determination 
of  the  path  that  a message  takes  through  the  network  is  controlled  by 
a routing  strategy;  i.e.,  a set  of  rules  that  guide  each  message  from 
its  source  to  its  destination.  The  path  is  not  known  in  advance. 
Instead,  each  node  determines  individually  onto  which  of  its  output 
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links  to  transmit  a message,  addressed  to  a specific  destination. 

This  determination  is  based  upon  network  connectivity,  network  con- 
gestion, and  link  failures.  Once  the  output  link  Is  selected,  a 
message  Is  placed  on  Its  queue  and  then  transmitted  over  the  link 
when  it  Is  free. 

In  an  operational  network,  the  routing  strategy  Is  usually  adaptive. 
It  uses  current  information  regarding  the  network  topology  and  the  user 
demands  and  Is  able  to  readjust  the  routing  logic  so  as  to  keep  the 
routing  decisions  near  optimum  at  every  Instant  of  time*  despite  the 
fact  that  the  network  status  Is  time  varying.  Adaptive  routing 
procedures  protect  against  network  failures  and  load  fluctuations 
and  are  essentially  for  traffic  management. 

Deterministic  routing  is  used  when  a network  is  designed.  It 
Implicitly  assumes  time  invariant  input  rates  and  a perfectly  reliable 
network  configuration.  In  general,  an  efficient  routing  strategy 
should  be  able  to  fully  utilize  network  capability  for  any  load  pattern 
by  sending  messages  on  minimum  delay  or  maximum  throughput  paths  and 
eventually  distributing  heavy  traffic  on  multiple  paths.  However,  the 
routing  strategy  cannot  prevent  network  congestion  and  a flow  control 
mechanism  is  needed  to  control  input  rates  before  congestion  occurs. 

The  single  most  significant  measure  of  performance  of  these  net- 
works is  the  average  time  for  a message  to  make  its  way  from  its 
source  node  through  the  network  to  its  destination  node.  We  refer 
to  this  as  the  average  message  delay.  This  report  is  essentially 
devoted  to  the  optimization  (minimization)  of  this  performance  measure 
for  existing  large  store-and-forward  computer  communications  networks. 
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In  Chapter  1,  which  is  a reorganization  of  existing  background 
material,  an  approximate  statistical  model  of  a store-and-forward  net- 
work is  given.  This  leads  to  a performance  measure  for  such  a network. 

In  Chapter  2,  which  presents  one  of  the  main  contributions  of  this 
thesis,  an  adaptive  routing  algorithm  Is  presented  for  handling  message 
traffic,  when  messages  are  transmitted  in  their  entirety.  Here  It  Is 
shown  that  the  average  delay  per  message  is  minimized  by  routing  long 
messages  through  direct  routes  and  short  messages  through  Indirect 
routes  as  much  as  possible.  This  is  accomplished  in  the  algorithm  by 
the  use  of  interacting  bias  factors  and  by  updating  of  node  delay 
i nformat ion. 

In  Chapter  3»  the  deterministic  routing  problem  is  summarized  and  an 
evaluation  of  the  adaptive  routing  algorithm  of  Chapter  2 is  presented. 

In  the  solution  of  the  deterministic  problem,  the  performance  function, 

Kth  mean  average  delay  per  message,  was  selected  In  such  a way  as  to 
approximate  the  function  of  the  adaptive  routing  algorithm. 

In  Chapter  l»,  a network  partitioning  approach  Is  suggested  for  large 
networks  in  order  to  be  able  to  apply  the  adaptive  routing  algorithm.  The 
main  contribution  here  is  a partitioning  algorithm  which  partitions  the 
network  topology  In  subnetworks  of  a specified  size. 

In  Chapter  5,  the  adaptive  algorithm  of  Chapter  2 is  modified  in 
order  to  "smooth  out"  updating  of  information  problems  arising  in  large 
networks.  Then  it  is  rewritten  in  detail  according  to  a partitioned 
network. 

In  Chapter  6,  some  new  network  topologies  found  in  the  literature 
are  examined  as  they  apply  to  the  adaptive  routing  problem. 


CHAPTER  l 


THE  MODEL  OF  A STORE-AND-FORWARD  COMPUTER  COMMUNICATION  NETWORK 
Introduction 

The  message  traffic  in  a store-and-forward  network  constitutes 
a stochastic  flow  problem.  In  such  a problem,  both  the  time  between 
successive  arrivals  to  the  system  and  the  demand  placed  on  the  link  by 
each  of  these  arrivals  are  random  quantities.  As  a consequence  of  that, 
a queue  may  form  during  stochastic  flow  even  if  the  average  flow  rate 
does  not  exceed  the  capacity  of  the  link.  A great  deal  of  investigation 
about  queueing  processes  of  this  type  has  been  done  as  early  as  1907. 

An  outstanding  contribution  was  made  by  Feller  [FELL  66]  through  his 
presentation  of  the  birth  and  death  process.  A message  arrival  in  the 
system  can  be  regarded  as  a birth,  and  a message  departure  (delivery) 
from  the  system  as  a death.  Queueing  network  models  as  they  apply  to 
computer  systems  analysis  have  been  analyzed  by  many.  A summary  of  such 
work  can  be  found  in  Busen's  thesis  work  [BUSE  71] • 

The  particular  case  of  the  store-and-forward  communications  network 
queueing  model  was  first  treated  by  Kleinrock  [KLEI  64],  His  model  has 
been  generally  accepted.  Until  today,  it  is  used  in  operational  networks 
for  the  determination  of  a performance  measure.  Several  have  tested  it 
by  simulation  [FULT  72],  [GERL  73A],  [KAMO  76]  and  all  agree  that  it  is 
quite  representative. 


in  this  chapter  Klelnrock's  model  Is  formulated  according  to  his 
analysis,  in  section  1.1  the  exact  mathematical  model  is  presented 
and  the  problems  associated  with  It  are  discussed.  Appropriate 
assumptions  are  introduced  In  order  to  derive  an  approximate  stat- 
istical model.  In  section  1.2  a performance  criterion  Is  defined  and 
analyzed,  and  in  section  1.3  a new  performance  criterion  Is  Introduced. 

1.1  The  Model 

The  mathematical  model  of  a store-and-forward  communication  network 
can  be  presented  in  terms  of  state  variables  [KLE I 64].  Such  a set  of 
variables  must  satisfy  two  conditions: 

(a)  it  must  include  (explicitly  or  Implicitly)  those  quantities 
which  are  of  interest;  example,  the  message  delay. 

(b)  The  set  must  be  complete,  or  closed,  such  that  knowledge  of 
the  state  variables  at  time  t and  knowledge  of  all  message  arrivals  from 
sources  external  to  the  net  in  the  closed  interval  (t,t')  are  sufficient 
to  specify  uniquely  the  state  variables  at  time  t'  a t.  This  second 
condition  describes  the  markovian  property. 

V/e  consider  a communication  network  with  N nodes  and  M one-way 
channels.  The  assumptions  made  in  order  to  arrive  at  the  model  are 

1.  External  Poisson  arrival  statistics 

2.  exponentially  distributed  message  lengths 

3.  infinite  storage  capacity  at  each  node 

4.  Single  destination  for  each  message 

5.  Full  reception  of  message  before  retransmission 

6.  h'oiseless  links  and  perfectly  reliable  nodes  and  links 
. Stationer ity  and  Independence  of  the  stochastic  processes. 
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It  is  clear  that  the  state  of  the  net  at  any  Instant  of  time 
must  include  detailed  information  as  to  the  number  of  messages  on  each 
queue,  the  length  of  each  message,  and  the  time  required  to  complete 
the  transmission  In  progress  on  each  link.  We  assume  that  each  message 
Is  labeled  with  an  origin,  a destination,  and  a priority.  Accordingly, 
we  define 

Cj  “ capacity  of  the  1th  link,  bits/sec 
n^  «*  number  of  messages  waiting  for  (or  being 
transmitted)  on  the  ith  link 
y.  = average  arrival  rate  of  messages  from  external 
sources  to  the  1th  link,  mes/sec 
Vin^Xln,yin  * messa9e  length.  In  bits,  of  the  nth  message 

waiting  for  (or  being  transmitted  on)  the  1th 
link,  whose  origin,  destination,  and  priority  are 
xin’  yir”  anc*  zln  resPect Ively I for  simplicity  of 
notation,  let  symbol  v^  denote  this  length. 

Vn  ■ set  of  numbers  (v  1 1 , v^,  ...,  v^  ) 

Tj  = time  remaining  to  complete  transmission  In  progress 
on  the  Ith  link. 

R ■ set  of  numbers  (r,,  r_,  ...,  r ) 

R + dt  = set  of  numbers  (r.+dt,  r.+dt,  ...,  r +dt) 

4 M 

; 

where  1 «*  1,2,...,H  and  n ■ 1,2, ...,n^ 

j 

The  state  of  the  network  at  any  time  t may  be  completely  des- 
cribed by  the  set  of  variables, 

r _ 

S " (Vn2 VVn,  ,Vn2  »'**»VnH'rl»r2 rM) 
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All  these  quantities  vary  with  time.  This  state  description  is  of 
unbounded  dimensionality,  since  the  variables  n^,  are  unbounded. 
Furthermore,  these  variables  are  necessary  in  order  that  the  state 
description  be  complete. 

For  each  state  S and  each  time  t a probability  density  function 
Pt(S)  is  associated  with  the  probability  that  the  net  will  be  found  In 
state  S at  time  t.  in  general,  it  would  be  desirable  to  find  an 
explicit  solution  for  the  function  p^ (S) . To  date  this  problem  remains 
unsolved.  A portion  of  the  analysis  will  be  carried  out  in  order  to 
indicate  the  source  of  the  difficulty.  Let  us  set  up  the  equations 
under  the  conditions 


ni  > 1 


0 < r.  < ^ 

i c. 


The  first  condition,  n > 1,  is  included  for  convenience.  The  end 
Vil 

points  zero  and  rr—  are  excluded  from  the  allowed  range  of  r.  in  order 

v ■ E 

i 

to  eliminate  temporarily  from  discussion  any  consideration  of  internal 

• i 1 

message  arrival s. 

For  this  case  we  write 


(n, ,n0, . • . ,n  ,V  ,V  ,...,V  ,R) 


t+dt ' i *2 


M"  ",  "2 


p (n . ,n ny ,V  ,V  ,...,V  ,R  + dt)(l  - E Y.dt) 
t I 2 H n,  n2  nM  M 1 


+ E Yj'dt  he  ni  Pt(n1,n2 n{-l, 


Vn  ,V  ,V  V ,R+dt) 

n,  n2  n,-l  nM 

^An  internal  message  arrival  occurs  when  a message  completes  its 
trcnsmi ss ion  between  tvc  nodes  internal  to  the  net  (as  opposed  to  an 
external  message  arrival  which  occurs  when  a message  arrives  at  its 
origin  from  a source  to  the  net). 
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where  Y ' represents  that  portion  of  y,  which  has  the  x,  , y,  , and 
i I I n j I n j 

z which  correspond  to  v.  , l/p  is  the  mean  value  of  the  message 
in  j in | 

length  exponential  distribution.  According  to  the  Chapman  Kolmogorov 
equation,  this  leads  to  the  following  partial  differential  difference 
equation: 


3pt 

3t~  (rVn2’-‘*’VVnf  ’Vn2 VnM’R) 


M 9pt 

£ n (n  t » • • • » ^14  » » V *•••*!/_  »R) 


1-1  9rI  v'i*"2* * ’ * ’"M* 'n j ' n,'  nM 


M 


i*  2* 

-MV, 


iij  VP®  Inf  Pt(ni,n2 V1 VVVn2 


V,-.' 


,V„  ,R) 


M 


M 


” j *no  » • • • »nu  *^n  »^n  ^ "f  J 

t i 2 M n?  n2  nM  i 


vii 


where  n.  > i and  0 < r.  < - — . The  equations  involving  r.  at  Its  end 
I I l | I 

points  force  one  to  consider  internal  message  arrivals  and  thus  become 

considerably  more  complex.  This  is  because  the  rules  of  the  routing 

procedure  must  be  included  in  order  to  determine  which  transitions 

occur.  The  task  of  solving  this  set  of  partial-differential  difference 

equations  is  enormous  and  nc  solution  has  yet  been  found. 

The  complexity  of  the  state  description  is  due  in  part  to  the 

constraint  that  each  message,  when  It  enters  the  net,  has  a permanent 

length  assigned  to  it.  The  message  maintains  this  same  length  as  It 

travels  through  the  net.  This  makes  it  necessary  for  the  variables 

V to  be  Included  in  the  state  description.  The  assignment  then  of  a 
nf 

permanent  message  length  not  only  increases  the  dimensionality  of  the 
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state  description,  but  also  complicates  the  stochastic  behavior  of  the 

net  by  Introducing  a dependence  among  some  of  the  random  variables  which 

describe  the  behavior  of  the  net. 

In  particular,  if  we  consider  two  successive  messages  arriving  at 

node  i from  some  other  node  internal  to  the  net,  then  the  interarrivai 

time  between  these  messages  is  not  independent  of  the  message  length 

2 

of  the  second  of  the  two  messages. 

More  specifically  a simple  two  node  tandem  net  is  considered  in 
Fig.  i.i  and  the  joint  probability  density  function  P^n»a£n^  *s 
derived  where 


v * message  length  for  nth  message,  bits 

a.  “time  between  arrivals  of  (n-i)st  and  nth  messages 
i n 

at  node  i 

Let,  for  convenience,  Cj  * i bit/sec.  By  assumption  the  message  length 
is  a random  variable  from  an  exponential  distribution,  the  arrivals 
probability  distribution  is  Poisson,  then  the  interarrivai  time  dis- 
tribution is  exponential. 

Then  let 

-ya  ln 

p(a,n)  " Ye 
-yv 

pl>n)  * ye 


figure  i.i  A two  node  tandem  network. 


2 

This  independence  exists  by  assumption  for  messages  which  arrive 
from  an  external  source. 


11 


Theorem  1 » 1 (due  to  Burke) 

The  steady-state  output  of  a queue  with  N channels  in  parallel  with 
Poisson  arrival  statistics,  and  with  lengths  chosen  Independently  from 
an  exponential  distribution  Is  itself  Poisson  distributed. 

For  the  case  considered  above,  Burke's  theorem  holds  and  since 
the  interdeparture  times  for  messages  leaving  node  1 are,  by  definition, 
identical  to  the  Interarrivals  times  for  messages  entering  node  2,  we 
see  that 


p(a2n)  “ Ye  n 

It  can  be  shown  [ KLE I 64]  that 

p(vn.a2n)  * P0„)p(a2„) 

which  illustrates  the  lack  of  independence  between v and  a.  . 

n Zn 

At  this  point  the  famous  independence  assumption  was  introduced 
by  [KLE I 64]  which  says  that 

Each  time  a message  is  received  at  a node 
within  the  network,  a new  length  v is 
chosen  for  this  message  from  the  following 
probability  density  function:  pi>)  **  pe 
It  Is  clear  that  the  assumption  does  not  correspond  to  the 
actual  situation  in  any  practical  communication  net.  But  simulation 
results  have  shown  that  the  independence  assumption  has  resulted  in 
rather  insign  If i cant  changes  In  the  average  message  delay  as  long  as 
the  network  is  highly  connected,  i.e. , there  are  multiple  paths 
emanating  from  every  node.  Then  the  behavior  of  the  message  delay 
for  a single  node  carrying  Internal  traffic  Is  very  much  the  same 
as  that  for  a node  supplied  exclusively  with  external  traffic. 


12 


Now  if  we  consider  only  fixed  routing,  we  recognize  that  the 
internal  traffic  flowing  into  each  channel  is  statistical ty  equivalent 
to  the  external  traffic  entering  the  net;  l.e.,  the  interarrlval  times 
and  message  lengths  are  independent  and  are  chosen  from  exponential 
distributions.  Thus,  each  link  satisfies  the  conditions  of  the 
single  exponential  channel.  This  fact  coupled  with  Burke's  [BURK  56] 
theorem,  allows  us  to  consider  each  link  (or  node)  separately  In  the 
mathematical  analysis.  Then  the  original  mathematical  models  have 
been  reduced  to  a network  of  queues  of  the  Jackson  type  [JACK  571. 
Baskett  and  Muntz  [BASK  72]  showed  that  each  queue  behaves  as  an 
H | H | I queue. 


1 .2  The  Average  Delay  T 

The  computer  communication  store-and-forward  network  Is  now 

decomposed  into  a network  of  independent  queues.  The  link  queues 

are  of  the  M|M|l  type.  Traffic  entering  the  network  then  from 

external  sources  forms  a Poisson  process  with  a mean  y (mes/sec)  for 

J * 

those  messages  originating  at  node  j and  destined  for  node  k.  The  total 
external  traffic  entering  the  network  is  defined  as 


N N 

Y = £ E Y„  (1.1) 

j = 1 k-1  IJ 

where  N Is  the  number  of  nodes  In  the  network.  Note  that  the  above 
total  external  traffic  definition  represents  a decomposition  of  the 
network  on  the  basis  of  origin-destination  pairs. 


By  fixed  routing  procedure  we  mean  that  given  a message's  origin 
and  destination,  there  exists  a unique  path  through  the  net  which  this 
message  rust  follow.  If  more  than  one  path  Is  allowed,  then  we  speak 
of  an  alternate  routing. 
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Since  each  link  in  the  network  is  considered  as  a separate  queue, 
we  define  as  Xj  the  average  number  of  messages  per  second  which  travel 
over  the  ith  link.  Then  the  total  traffic  within  the  network  is  redefined 
as 

M 

X - Z X.  (1.2) 

i = l ' 


where  M is  the  number  of  links  in  the  network.  This  definition  represents 
a decomposition  down  to  the  single  link  level. 

A very  Important  performance  measure  for  store-and-forward  com- 
munications nets  Is  the  average  message  delay  [ KLE I 76] 


T *•  Elmessage  delay]. 
Define  first  the  quantity 


” E [message  delay  for  a message  whose  origin 
is  j and  destination  k] . 


Then  the  last  two  quantities  are  related  by 


N N Y 

T = l l 


j = 1 k= 1 


Y Zjk 


(1.3) 


since  the  fraction  Y../Y  of  the  total  entering  message  traffic  will 

J k 

suffer  the  delay  Z on  the  average. 

Let  us  now  denote  the  path  taken  by  messages  that  originate  at 

node  j and  are  destined  for  node  k (the  "j-k  traffic")  by  We  say 

that  the  ith  link  (of  capacity  C.)  is  included  in  the  path  ir.,  if  that 

i j k 

link  is  traversed  by  messages  following  this  path  and  symbolize  it  by 
i : C j It  Is  clear,  therefore,  that  the  average  message  flow  rate 

on  link  i,Xj,  must  be  equal  to  the  sum  of  the  average  message  flow  rates 
of  all  paths  that  Include  this  link,  that  is 
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Xi  = Yjk 
J k 

j , k : C . e it  . . 
J ' i jk 


(1.4) 


Let  us  now  define  the  delay  of  the  individual  link  i as 
Tj  = Eftime  spent  by  messages  waiting  for  and 
using  the  1th  1 ink] 

Thus  Tj  is  just  the  average  time  "in  system"  where  the  system  Is 
defined  as  the  ith  link  plus  the  queues  of  messages  In  front  of  that 
link.  We  may  also  write 


Zjk 


£ T. 

T:C,  Oj' 


From  equations  (1.3),  (1.5)  we  therefore  have 


liis 


j=l  k=l 


,:Ci  £V 


(1.5) 


After  interchanging  the  summations  order  we  obtain 


M 

T = E 
i-i 


T. 

— £ 

Y 


. J K jk 

j »ks Cj  e 7T j ^ 
Using  equation  (1.4)  we  have 


T - 2 T T, 
1 = 1 T 


(1.6) 


(1.7) 


We  have  thus  decomposed  the  average  message  delay  Into  its  single 
1 i nk  delays  T. . 

We  now  have  to  solve  for  the  message  average  delay  cn  a link.  The 
ith  link  is  now  representable  as  an  M | H | 1 system  with  Poisson  arrivals 
at  a rate  X.  and  exponential  service  times  (message  transmission  times) 


l 
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of  mean  1/pCj  sec.  The  solution  for  Is  given  then  by 
„ _ 1 


1 


“i ' xi 


(1.8) 


and  so  equation  (1.7)  becomes 
M A, 


'1 


* ~ [ 
1=1 


Y 1 “ A 


] 


(1.9) 


Letting  A./p  = f^  average  bit  rate  on  link  1 (blts/sec),  we  obtain 

M ft 

(1.10) 


1 r ~i 
v j=i  ci  ‘ fi 


This  is  the  main  queuing  analysis  result  for  the  store-and-forward 
communications  net. 

In  high-speed  networks.  It  may  be  Important  to  Include  the 
propagation  delay  P.  which  Is  the  time  required  for  the  energy 
representing  a single  bit  to  travel  down  the  length  of  the  1th  link. 
It  may  also  be  of  importance  to  Include  the  factor  k defined  as  nodal 
processing  time.  In  the  work  to  follow,  we  assume  Pj  = 0 and  k = 0. 


1 .3  Mean-Kth-Power  Delay  T^ 

It  was  observed  by  [ME I S 72]  that  In  minimizing  the  average 
delay  T as  from  equation  (i.7)  a wide  variation  was  allowed  among 
the  i ink  delays  Tj  and,  therefore,  among  message  delays  between 
different  source-destination  pairs.  In  order  to  take  into  account 
such  variation,  an  alternate  performance  measure  was  proposed: 


.(K) 


M A. 

r e -7 

i=i  Y 


(T , ) K1 


i/K 


(1.11) 


where  K > 1 Is  a finite  number.  If  K is  sufficiently  large,  the 
variations  among  message  delays  are  considerably  reduced. 


CHAPTER  2 


ADAPTIVE  STRATEGY  TECHNIQUE 

Introduct ion 

In  an  adaptive  strategy  the  selection  of  the  output  channel  at 
node  K depends  on  the  estimate  of  the  delays  that  a message  would 
suffer  when  transmitted  over  different  output  channels.  Adaptive 
algorithms  are  mostly  suited  to  operational  networks  because  of 
their  ability  to  adjust  to  different  traffic  loads  and  network 
failures.  In  this  chapter  an  adaptive  routing  algorithm  will  be 
developed  applicable  to  store-and-forward  computer-communications 
networks. 

Extensive  work  on  adaptive  routing  techniques  has  been  done 
by  Fultz  [FULT  72].  He  considered  store-and-forward  packetized  net- 
works, like  the  ARPA  net.  The  algorithm  to  follow  Is  based  on  his 
work.  A quadratic  routing  scheme  is  presented  by  [AGNE  75]  where  a 
quadratic  bias  factor  encourages  a lot  of  alternate  routing.  Recent 
work  was  published  by  Kamoon  [KAMO  76]  where  he  treats  the  network 
design  problem  according  to  adaptive  routing  consideration. 

In  section  2.1  a definition  of  the  adaptive  routing  problem 
is  stated  and  a classification  of  the  adaptive  routing  techniques 
is  presented. 

In  section  2.2  an  adaptive  routing  algorithm  is  developed. 

The  algorithm  is  applied  to  an  eight  node  network. 
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fn  section  2.3  the  simulation  language  used  for  the  computer 
simulation  Is  discussed.  The  simulation  system  characteristics 
along  with  the  drawn  conclusions  are  presented. 

2. I The  Adaptive  Routing  Problem 

A general  definition  of  the  adaptive  routing  problem  was  first 
stated  by  Klelnwork  [KLEIN  6^*]  as  follows: 

minimize  T (the  average  message  delay) 

over  the  /'link  capacity  assignments 
variables  message  priority  discipline 
‘ routing  doctrine 
topology 

subject  to  a suitable  performance  criterion  and 
external  traffic  requirement. 

It  Is  desirable  to  classify  network  adaptive  routing  techniques 
In  order  to  gain  Insight  Into  their  structure  complexity  and  perfor- 
mance. Such  classification  has  been  done  by  Prosser  [PROS  62],  Kleln- 
rock  [KLEI  61*],  Fultz  [FULT  72],  and  Gerla  [GERL  73],  and  others.  A 
summary  of  all  these  classifications  Is  presented  by  McQuillan 
[MCQU  7*0. 

The  two  major  categories  are  found  under  the  title  of  adaptive 
deterministic  pol  Icles  and  adaptive  stochastic  pol  Ides.  Fultz  has 
presented  ttoem  In  a concise  and  accurate  manner  as  follows: 


w 
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Adaptive  routing  algorithm  classification 
1 . Deterministic  techniques 


* Flooding 


ail 


sel ective 

* Fixed 

* Network  routing  control  center  (NRCC) 

present 

* ideal  observer  scheduling  problem. 

\ future 

2.  Stochastic  techniques 

.asynchronous  update 
"'^periodic  update 
.shortest  queue  + bias 


* Distributed: 


* Isolated 


* Random 


local  delay  estimate 


2.1.1  Deterministic  Techniques 

The  four  basic  deterministic  techniques  are: 

Flooding.  Each  node  receiving  or  originating  a message  transmits  a 

copy  of  it  over  "all"  outgoing  links,  or  over  a set  of  "selective" 

outgoing  links;  this  transmission  occurs  only  after  the  ncde  has 
checked  to  see  that  it  has  not  previously  transmitted  the  message,  or 

that  it  is  not  the  destination  of  the  message.  This  technique  has  been 

discussed  by  6oehm  and  Madiey  [BOEH  66].  Their  conclusion  is  that  the 
inefficiency  of  this  technique  Is  tolerable  if  one  has  only  a few  messages 
to  deliver.  However,  a large  voJume  of  communications  traffic  neces- 
sitates more  efficient  techniques.  Another  drawback  to  this  technique 


Is  that  each  node  requires  a mechanism  to  recognize  previously  trans- 
mi  tted  messages. 

Fixed  routing.  Fixed  routing  algorithms  specify  a unique  path 
(N£,.i.,Nq)  followed  by  a message  which  depends  only  upon  the  source- 
destination  node  pair  (Nj»Nq).  To  accomplish  this,  each  node  has  a 
routing  table.  Depending  on  the  message  destination  each  entry  in  the 
routing  table  contains  the  next  unique  node  In  the  message's  path. 
Kieinrock  [ KLE I 64],  [ KLE I 76]  and  Prosser  [PROS  62]  have  examined 
several  of  these  techniques.  Fixed  routing  techniques  require  completely 
reliable  nodes  and  links.  However,  they  do  allow  for  hfgh  efficient 
high  volume  traffic  flow  and  are  very  stable.  (I.e.,  they  are  relatively 
unaffected  by  small  changes  In  the  network  topology.) 

Network  routing  control  cehter  (NRCC).  With  this  technique,  one  of  the 
network  nodes  Is  designated  as  the  NRCC.  This  center  collects  perfor- 
mance information  about  the  network  operation  and  computes  routing  tables 
for  each  node  in  the  network.  Computation  of  the  routes  by  NCRR  is  done 
on  a global  basis  and  this  insures  loop-free  paths  between  ail  source- 
destination  node  pairs.  Thus,  a fixed  routing  procedure  Is  maintained 
between  NRCC  updates. 

There  are  a number  of  drawbacks  to  this  technique.  By  the  time 
the  nodes  begin  using  the  new  routing  tables,  the  performance  infor- 
mation that  was  used  in  the  computation  of  the  routing  tables  may  be 
cut  of  date  in  relation  to  the  current  state  of  the  network.  In 
addition,  transmission  costs  and  vulnerability  become  significant 
considerations. 


Ideal  Observer  routing.  This  technique  Is  essentially  a scheduling 
problem.  Each  time  a new  message  enters  a node  Its  route  Is  computed 
to  minimize  Its  travel  time  to  Its  destination  node,  based  upon  the 
complete  present  Information  about  the  messages  already  In  the  network 
and  their  known  routes.  If  the  Ideal  observer  has  Information  about  the 
occurrence  of  future  events,  then  this  Information  could  also  be 
utilized  In  the  computation  of  the  route.  This  technique  Is  obviously 
Impractical  for  an  operational  network  but  theoretically  provides  the 
minimum  average  message  delay. 

2.1.2  Stochastic  Techniques 

The  three  basic  stochastic  techniques  are: 

Random  routing.  Random  routing  procedures  are  those  decision  rules 
In  which  the  choice  as  to  the  next  node  to  visit  Is  made  according  to 
some  probability  distribution  over  the  set  of  neighbor  nodes.  The  set 
of  neighbor  nodes  utilized  In  the  decision  rule  can  be  "all"  of  the 
connected  nodes  or  can  be  based  "selectively"  over  that  set  of  nodes 
which  are  In  the  general  direction  of  the  message  destination.  Kleln- 
rockJfkLEI  64]  and  Prosser  [PROS  62]  have  Investigated  numerous  random 
routing  techniques  and  have  shown  that  they  are  highly  Inefficient  In 
terms  of  message  delay,  but  are  extremely  stable. 

Isolated  and  distributed  techniques.  All  of  the  Isolated  and  dis- 
tributed routing  algorithms  operate  In  basically  the  same  manner.  A 
delay  table  is  formed  at  every  node  whose  entries  are  the  estimated 
delays  to  go  from  the  node  under  consideration  to  some  destination  node. 
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The  manner  In  which  these  estimates  are  formed  and  updated  depends 
upon  the  specific  structure  at  the  routing  algorithm. 

In  the  shortest  queue  + zero  bias  a message  route  Is  selected  by 
placing  it  in  the  shortest  output  channel  queue.  This  is  essentially 
Baran's  Hot  Potato  routing  concept  [BARA  61*].  Since  the  route 
selected  is  Independent  of  the  message  destination,  the  delay  table 
requires  information  only  on  the  output  link  queues  of  the  sending 
node. 

In  the  local  delay  estimate  algorithm,  a message's  route  Is 
selected  via  the  delay  table  estimates.  These  estimates  are  formed 
from  information  such  as  the  time  the  message  has  spent  in  the  network 
traveling  from  its  source  node  to  the  current  node  and  backwards 
learning  delay  Information  from  the  reverse  outgoing  line  corresponding 
to  the  forward  line  of  the  full-dublex  pair  which  the  message  is 
about  to  enter.  This  backwards  learning  has  been  investigated  by 
Baran  [BARA  64],  Boehm  [BOEH  64]. 

In  the  distributed  routing  techniques  classification,  ail  routing 
algorithms  utilize  the  same  basic  techniques  to  compute  and  update 
the  delay  table  estimates,  but  the  instants  at  which  these  tables  are 
updated  and  the  route  selection  procedures  depend  upon  the  particular 
structure  of  the  algorithm. 

Of  ail  the  stochastic  techniques,  the  distributed  routing  is  the 


i 


most  efficient  for  handling  line  and  node  failures.  Chce  a failure  Is 
determined,  the  proper  delay  table  entries  in  every  node  can  be  forced 
to  remain  excessively  large  as  long  as  the  failure  persists. 
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We  felt  that  the  distributed  stochastic  routing  techniques  have 
the  best  potential  performance  to  offer  to  operational  store-and-forward 
computer-communication  networks. 

The  adaptive  algorithm  that  follows  here  Is  of  the  distributed 
stochastic  type  based  on  Fultz's  work  for  the  Arpanet. 

?.2  Adaptive  Routing  Algorithm 

There  are  two  ways  to  Implement  a store-and-forward  network.  In 
the  one.  each  message  Is  broken  up  Into  submessages  ("packets")  of  equal 
length  which  are  routed  as  individual  messages  and  then  reassembled  at 
the  destination.  In  the  other,  each  message  Is  routed  In  Its  entirety 
with  consequent  variation  In  message  length.  The  algorithm  Is  going 
to  deal  with  the  message  In  Its  entirety. 

In  the  algorithm  , each  node  In  the  network  makes  Its  decisions 

I 

on  the  basis  of  Information  sent  by  nodes  to  which  the  deciding  node 
Is  directly  connected.  Since  It  In  turn  sends  Information  to  all  such 
nodes,  the  Information  can  "percolate"  through  the  system.  Each 
piece  of  control  data  Is  sent  only  from  one  node  to  the  next,  but  In 
order  to  ensure  rapid  flow  of  Information,  the  data  must  be  sent 
fairly  frequently,  and  thus  the  control  data  constitutes  the  bulk  of 
the  non-message  data  flow. 

] 

The  optimization  problem  Is  essentially  a trade-off  of  control 
data  flow  against  Improvement  In  message  delay.  This  Is  because  the 
message  delay  itself  is  a function  of  the  data  flow  since  the  data  are 
treated  as  messages  and  therefore  Increase  the  message  load. 
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2.2.1  The  Routing  Algorithm 

Each  node  maintains  a delay  table  as  shown  in  FIg.2j.The  table 

has  as  many  rows  as  there  are  destination  nodes  and  as  many  columns  as 

there  are  output  links  from  the  node.  An  entry  ^(D.L^)  Is  the  estimated 

delay  to  go  from  node  j (the  "deciding"  node)  to  a destination  node  D 

using  link  L as  the  next  step  on  the  path  to  D.  The  routing  table  is 
m 

formed  from  the  delay  table  by  choosing  for  each  row,  the  link  which  has 

the  minimum  delay  for  that  row  (destination).  If  the  routing  table  entry 

for  the  jth  node  is  R,(D,L  ) then  R,  (D,L')“  i where  L'  corresponds  to 

j m j m m 

the  link  with  min  T,(D,L  ).  The  heart  of  the  aigorithm  is  the  technique 

V 1 m 


by  which  the  delay  table  entries  are  changed.  A given  node  receives 
delay  estimates  from  each  node  to  which  It  Is  directly  connected.  The 
given  node  then  forms  delay  estimates  of  Its  own,  utilizing  this  Infor- 
mation, and  transmits  these  estimates  In  Its  turn.  In  this  way,  the 
estimates  "percolate"  through  the  system.  Suppose  a given  node  J has 
made  a decision  to  inform  its  "neighbor  nodes"  (l.e.,  those  nodes  with 
a direct  link  to  node  J)  of  its  estimated  minimum  current  delay  to  all 
other  nodes  in  the  network.  Switch  J forms  a Delay  Vector  Vj  which  has 
one  entry  T ( I ) for  each  other  node  In  the  network,  where 


T(i ) «=  min  T.(i,L  ) 
f J m 
m 


(2.1) 


minimized  over  the  set  Lm  of  output  links  from  node  J.  This  Delay  Vector 
is  then  transmitted  to  the  neighbor  nodes.  When  a switch  K receives  a 
delay  vector  from  node  J,  node  K adds  the  queue  delay  (in  bits)  on  the 
node  J outbound  link  to  each  entry  in  the  Delay  Vector  (plus  an 
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SAMPLE  CONTROL  TABLES  FOR  NODE  4 


D 

L1 

Delay  table 

L2  l3 

L4 

D 

Minimized 
delay  vector 

Delay  (Lm) 

1 

10 

12 

15 

27 

1 

10 

(Lx) 

2 

14 

16 

7 

33 

2 

7 

(l3) 

3 

8 

19 

13 

29 

3 

8 

(Lx) 

5 

26 

23 

18 

24 

5 

18 

(L3> 

6 

21 

25 

17 

15 

6 

17 

(l3) 

7 

15 

18 

26 

23 

7 

15 

(Lx) 

8 

32 

29 

24 

13 

8 

13 

(l4) 

Routing  table 

D 

L_1 

h 

h 
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1 

0 

0 

0 

2 

0 

0 

1 

0 

3 

1 

0 

0 

0 

5 

0 

0 
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0 

6 

0 

0 

1 

0 

7 

1 

0 

0 

0 

8 

0 

0 

0 

1 

The 

notation  refers 

to  Fig.  2.3 

(p.  33). 

D Is  the  destination 

node. 

Figure  2.1  Delay  table  and  delay  vector  stored  In  a node 
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adjustment  factor  DELTA  explained  below)  and  replaces  the  L^  column  in 
its  delay  table  with  the  new  values.  The  delay  table  equation  then  Is 

T.  (D,L  ) - queue  (K,L  ) + T(D)  + DELTA  (2.2) 

* 9 9 

where  queue  (K.L^)  Is  the  outgoing  queue  of  node  K on  link  Lg.  T(D)  is 
the  entry  of  the  delay  vector  for  destination  D.  The  adjustment  factor 
depends  on  the  type  of  network.  In  a packet  I zed  system,  DELTA  is 
usually  a constant  related  to  the  average  transmission  time  of  a packet. 

If  the  tables  are  updated  rapidly,  then  the  delay  tabie  estimate  in 
node  K will  have  a term  approximately  equal  to  (N*(K,D,Lg))x(DELTA) 
where  N*(K,D,L^)  is  the  number  of  links  encountered  on  the  path  between 
node  K and  node  D through  link  L^;  i.e.,  if  DELTA  is  ciose  to  the 
average  message  length  and  if  we  have  frequent  updating  and  if  the 
queues  are  short,  then  the  term  N*(K,D, Lg)xDELTA  approximates  the 
delay  caused  by  the  transmission  of  the  message  on  the  intervening 
links.  For  variable  length  message  systems  this  is  even  more  significant. 
Since  the  message  is  accumulated  at  each  node,  there  should  be  a "penalty 
term"  for  traversing  a circuitous  path.  Such  a path  would  cause  delay 
which  is  not  represented  by  the  existing  queue  lengths,  in  the  asyn- 
chronous update  for  very  long  messages  explained  below,  this  term  is 
calculated  directly  using  the  actual  message  length.  However,  this  is 
not  the  only  consideration  in  choosing  DELTA.  The  routing  scheme  is 
suboptimai  and  constantly  being  updated.  Thus,  there  is  a real  poss- 
ibility of  a message  traversing  a "ioop"  and  passing  through  the  node 
which  initiated  it,  and  this  obviously  will  hurt  the  average  delay. 

There  is  no  easy  way  to  prevent  this  completely  and  as  long  as  the 
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average  delay  Is  reduced,  occasional  looping  Is  tolerable.  The 
chances  of  looping  are  obviously  increased  if  the  message  follows  a 
circuitous  path,  and  the  value  of  DELTA  can  be  adjusted  to  Introduce 
an  additional  penalty  on  circuitous  routes  in  order  to  reduce  looping. 
The  exact  value  of  DELTA  which  gives  the  minimum  average  delay  must 
be  determined  by  simulation. 

For  a non-packet  I zed  system,  the  algorithm  does  not  account  for 
the  wide  variation  in  message  lengths.  In  the  normal  operation,  the 
constant  DELTA  should  be  at  least  as  large  as  the  average  message 
length.  This  will  force  very  short  messages  to  take  "more  direct" 
but  actually  slower  routes,  but  this  should  not  be  a serious  problem. 

For  extremely  long  messages,  the  system  should  have  a much  stronger 
bias  towards  the  "most  direct"  route,  and  this  Is  built  Into  the 
algorithm  In  the  form  of  a special  routing  technique  for  long  messages. 
When  an  incoming  message  length  exceeds  a threshhold,  the  routing  Is 
not  done  via  routing  tables,  but  is  done  dynamically  via  the  following 
formula: 

L ■ min  {queue  ( 1 , L ) + N (l,D,L  )x£.  - N (l,D,L  )x)l  +T(D)} 

^ m m m avg 

m 

(2.3) 

where  H = actual  message  length;  &aVg  average  message  length.  In  this 
formula,  the  instantaneous  queue  is  used,  but  the  bias  term  multiplies 
the  number  of  links  in  the  shortest  path  times  the  length  of  the  message 
and  subtracts  the  estimated  delay  caused  by  the  fixed  bias  term  In  the 
average  message  routine.  This  must  be  done  explicitly  since  the  delay 
estimates  which  are  received  from  other  nodes  do  not  take  account  of 
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this  very  long  message.  The  Inclusion  of  this  term  gives  a very 
strong  bias  towards  the  shortest  route;  but  if  the  queues  on  this 
route  are  long  enough,  another  route  can  be  taken  If  it  appears  to 
have  less  delay. 

A simple  algorithm  can  be  added  to  increase  survivability  by 
performing  automatic  failure  detection.  By  keeping  track  of  the  number 
of  links  traversed  by  an  incoming  message,  the  receiving  node  could 
determine  network  failures  on  the  basis  of  traffic  data,  as  described 
in  [TORB7*0.  When  a failed  link  or  node  is  detected,  either  by  the 
node  or  by  central  control,  the  delay  entry  for  that  path  can  be 
made  so  large  that  the  node  algorithm  will  not  attempt  to  route 
messages  over  the  failed  equipment. 

The  update  of  the  delay  tables  Information  Is  done  as  follows. 
Periodically,  new  estimates  received  from  a neighbor  node  replace  the 

older  estimates.  There  are  two  approaches  to  update  frequency.  One 

1 

sends  the  update  data  at  regular  inte'rvais  (synchronous  update);  the 
other  sends  the  data  only  "as  required"  (asynchronous  update).  In  the 
case  of  a network  (such  as  a military  one),  a prime  consideration  Is 
reliability  and  the  speedy  detection  of  faults  which  are  enhanced  by 
synchronous  update.  Thus,  the  algorithm  includes  a synchronous  update. 

The  assignment  of  a very  long  message  to  a queue  will  result  in 
a large  enough  delay  to  justify  extra  Information  transfer.  Thus, 
an  asynchronous  update  is  included  in  the  algorithm  and  In  this  update 
is  triggered  by  the  use  of  the  long-message  routing  algorithm  in  the 
node. 

\ 
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Figure  2.2 


Flowchart  of  a message  routing  the  network. 
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Cne  can  also  include  a "spill-over"  asynchronous  update  when  an 
asynchronous  delay  vector  is  received  at  a node  and  the  delay  vector 
causes  an  over-threshhold  change  in  a delay  table  entry.  The  effect 
on  average  delay  of  various  message-length  threshholds  and  message- 
length  traffic  mixes  must  be  evaluated  by  simulation.  A flowchart  of 
a message  routing  through  the  net  is  shown  in  Figure  2.2. 

2.3  The  Simulation  Language 

Of  the  several  languages  available,  the  language  ASPOL  was  chosen 
for  the  simulation.  Originally  developed  to  simulate  multi-processing 
systems  in  computers,  [ASPOL  72]  has  several  features  which  make  it  con- 
venient foi  the  simulation  of  communication  networks.  ASPOL  is  a general- 
purpose,  process-oriented  language  based  on  process  communication  and  con- 
trol structures  used  in  computer  operating  systems.  The  language  is  based 
cn  the  language  SOL  developed  by  D„  E.  Knuth  et.  al„  [KNUT  68],  and  it  is 
ALGOL-like.  Among  the  major  features  of  ASPOL  are:  Language  facilities 
to  concisely  describe  systems  of  interacting  processes;  the  ability  to 
define  and  manipulate  sets  of  entities,  such  as  links;  and  macro  facil- 
ities, which  can  extend  the  language  to  specialized  problems  such  as 
communication  systems.  ASPOL  is  defined  in  terms  of  "processes," 
"facilities,"  and  "events."  A process  is  a particular  example  or 
instance  of  a set  of  activities,  in  the  simulation,  each  message  is  a 
process,  and  the  set  is  al'  messages  being  transmitted  at  a given  time. 

2.3-1  Description  of  the  Simulation 

An  eiqht  node  store-and- forwa rri  communication  system  was  simulated 
using  ASPCL  on  the  CDC  6500  computer.  The  basic  icea  of  the  simulation  Is: 
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I 

A global  program  simulates  the  routing  decisions  of  the  individual 
nodes,  but  makes  decisions  only  on  the  basis  of  information  local  to 
each  node.  The  global  program  is  called  SiH  and  has  specialized  sub*- 
routines  to  carry  out  the  actual  computations.  The  data  for  the  entire 
network  is  stored  in  global  arrays  and  SiH  makes  local  decisions  on 
the  basis  of  Information  selected  from  the  arrays.  The  data  transfer 
which  updates  the  arrays  is  simulated  by  a 
special  process  called  UPDATE.  The  system  information  which  Is  the 
object  of  the  simulation  is  collected  from  the  operation  of  UPDATE. 

This  process  is  initiated  at  fixed  intervals  whose  length  can  be 
varied,  giving  the  synchronous  update  capability  and  also  at  variable 
Intervals  determined  by  SIM  which  simulates  the  asynchronous  update. 

The  messages  are  simulated  as  particular  examples  of  a process  MESSAGE, 
where  each  example  has  a length,  arrival  time,  and  origin  and  destination 
nodes  drawn  from  appropriate  probability  distributions.  Each  message 
Is  routed  by  routing  tables  local  to  a given  nod2  (recalculated  for 
long  messages)  and  assigned  to  the  appropriate  outgoing  link,  where 
it  is  queued  as  needed  by  the  simulation  translator;  the  chosen  link 
is  reserved  for  a period  corresponding  to  the  transmission  time,  then 
the  intermediate  node  Is  assigned  as  the  new  Initial  or  origin  switch 
and  the  program  loops  back.  When  long-message  recalculation  takes 
place,  the  new  minimum  Delay  Vector  Is  "transmitted"  to  the  entries 
for  the  neighboring  nodes,  but  the  asynchronous  update  stops  there. 

It  can  be  extended  to  a "percolation"  exchange  if  desired. 

The  SIM  Program 


The  SIM  program  Is  the  master  program  which  initiates  the 
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messages,  determines  the  length  of  the  simulation  and  the  statistics 
of  the  random  parameters,  and  collects  data  on  the  operation  of  the 
system  upon  completion  of  the  simulation  run.  The  heart  of  the  program 
is  the  while  loop;  this  determines  the  length  of  the  run  In  simulation 
time,  and  during  the  run  It  generates  messages  with  pseudo-random 
parameters  (arrival  time,  length,  and  nodes)  and  Initiates  the  periodic 
updating  of  the  arrays.  The  only  tricky  part  Is  performing  the  periodic 
update.  The  ASPOL  program  provides  a TIME  parameter  which  represents 
simulated  time,  but  this  Is  not  really  a continuous  parameter.  TIME 
Is  advanced  by  process  "hold"  Instructions,  which  In  the  SIM  program 
represent  the  arrival  Intervals  and  In  the  MESSAGE  process  represent 
the  transmission  delays.  ASPOL  takes  account  of  the  Initiation  times 
and  ail  "hold"  periods  generated  by  particular  messages,  and  advances 
the  Internal  TIME  paramter  as  these  "holds"  are  executed  In  "chronoglcal" 
order;  thus  TIME  takes  on  only  those  values  which  occur  at  the  end  of 
"hold"  periods.  Although  in  a "loaded"  network  there  would  be  enough 
message  arrivals  and  retransmissions  so  that  the  TIME  parameter  would 
assume  a fairly  dense  set  of  values,  nevertheless  the  density  would 
vary,  and  thus  the  synchronous  update  is  initiated  by  a repetitive 
"hold"  period  rather  than  by  "clock-watching."  The  program  has  to 
Interleave  the  pseudo-random  arrival  "holds"  and  the  synchronous 
update  "holds,"  and  this  Is  accomplished  by  the  nested  conditional 
statements. 

The  UPDATE  Process 

The  UPDATE  Process  is  not  really  a simulated  process,  since  there 
Is  theoretically  only  one  example  In  execution  at  any  time,  but  this 
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format  Is  convenient.  In  the  actual  system,  the  update  Is  executed 
Independently  In  each  node.  The  update  calculations  performed  by  the 
process  have  been  exactly  explained  In  the  description  of  the  algorithm. 

The  Message  Process 

The  MESSAGE  Process  does  most  of  the  simulation  work.  Each  sim- 
ulated message  through  the  network  Is  associated  with  an  example  of  the 
MESSAGE  process,  and  at  any  point  In  simulated  time,  many  such  examples 
exist.  Each  example  has  the  following  parameters  associated  with  It: 
the  message  length,  the  Initial  node,  the  destination  node,  and  the 
time  of  arrival  at  the  Initial  node.  These  parameters  are  selected  from 
pseudo-random  function  routines.  In  the  Initial  simulation,  the  length 
of  each  message  Is  selected  from  an  exponential  distribution  with  mean 
equal  to  the  observed  mean  of  present  AUTODIN  traffic.  The  Initial  node 
and  destination  node  are  selected  from  flat  distributions  (all  nodes 
equally  likely)  and  the  time  of  arrival  at  the  selected  Initial  node 
Is  selected  from  a Poisson  distribution  whose  parameters  can  be  varied 
to  change  the  system  traffic  load.  These  assumptions  seem  reasonable 
for  the  first  steps.  Later  work  would  vary  the  selection  of  origin 
and  destination  nodes  to  reflect  the  actual  traffic  Inputs  to  the 
network  at  specific  nodes  and  would  permit  varying  the  arrival  times 
at  each  node  Independently. 

I 

2. 3 . 2 Simulated  System  Characteristics 

The  system  simulated  Is  AUTODIN  In  Europe  [Appendix  I ].  With 
the  CONUS  gateways,  the  system  has  8 nodes  and  32  links  as  shown  In 
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Fig.  2-3  All  links  are  two  way.  The  node  connectivity  varies  from  3 
to  5,  and  each  node  represents  a switch.  A detailed  description  of 
the  simulation  Is  contained  In  a Purdue  University  Technical  Report 
[HOUS  753. 

The  values  of  the  following  parameters  are  drawn  from  probability 
distributions: 

1.  Message  length:  Exponential  distribution  with  mean  of  9 
seconds,  derived  from  actual  message  statistics. 

2.  Arrival  time:  Assumed  Poisson  stochastic  process  so  that 
interarrivai  delay  Is  exponentially  distributed  with  average  ■ 
1/.88  * 1.14.  This  Is  derived  from  an  average  of  200,000 
messages  per  day,  38  percent  of  which  are  assumed  to  be 
Inter-switch  traffic,  giving  a rate  of  .88  messages/sec. 
(arrival  rate). 

3.  Initial  and  destination  nodes:  Independently  drawn  from  a 
distribution  uniform  over  the  Integer  labels  of  the  nodes. 

The  simulation  was  run  several  times  for  varying  values  of: 
synchronous  update  period,  message  length  threshhold  for  Initiation 
of  asynchronous  update  (special  routing)  and  the  bias  factor  DELTA. 

The  mnemonics  UPDATE,  TRSLD,  and  DELTA,  respectively,  are  used  for 
these  quantities  in  the  presentation  of  the  results. 

The  variable  UPDATE  20,  40,  etc.  means  that  the  synchronous 
update  process  Is  being  Initiated  every  time  20,  40  messages  etc. 
have  arrived  In  the  network. 
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The  update  Information  messages  that  the  nodes  send  to  each 
other  after  update  takes  place  are  accounted  for  by  holding  every 
link  In  the  network  for  a short  period  of  time  (.007  sec.  in  this 
case)  after  the  update  calculations  are  over. 

For  each  run,  the  delay  for  each  message  was  measured  and 
the  average  delay  computed  using  the  sample  mean  formula. 

2.3.3  Conclus Ions 

Simulation  results  are  presented  here  using  the  Autodin  network 
as  an  example.  Data  for  two  or  three  traffic  loads  were  taken,  when 
the  inter-arrival  time  variable  takes  the  values  of  1/y  = 1.14, 
which  represents  38%  of  the  overall  Autodin  traffic  (76000  mes/day) , 

1/y  = .97  which  represents  44.6%  of  the  traffic  (89000  mes/day),  and 
1/y  “ .9  which  represents  **8.1%  of  the  traffic  (96200  mes/day).  Data 
are  presented  for  1/2  hour  simulated  time. 

In  Fig. 2.4,  the  RATIO  of  the  ACTUAL  TOTAL  AVERAGE  DELAY  PER  MESSAGE 
over  the  IDEAL  TOTAL  DELAY  PER  MESSAGE  (ideal  delay  per  message  Is 
taken  to  be  the  transmission  time  of  the  message  following  the  direct 
rou to  to  its  destination  if  there  were  no  queues  In  the  system)  is 
plotted  versus  the  bias  factor  DELTA  for  various  update  periods  and 
with  a fixed  special-routing  (asynchronous  update)  threshhold  of  12. 
Three  traffic  loads  are  being  Investigated  for  1/y  = 1.1*1,  1/YB  .97, 

1/y  ■ .9  In  Fig.  2.4a,  Fig.  2.4b,  and  Fig.  2 ,4c,  respectively. 

With  a low  threshhold  of  12,  slightly  over  the  average  of  9,  a 
large  portion  of  the  messages  are  special-routed.  The  effect  of 
DELTA  is  to  bias  the  network  towards  choosing  the  shortest  path. 
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The  best  results  for  all  three  traffic  loads  were  obtained  for 
the  fastest  update  of  20  and  low  values  of  DELTA.  Low  values  of  DELTA 
would  result  In  low  encouragement  of  the  direct,  route  for  messages  whose 
length  is  below  the  threshold  value. 

When  the  update  Is  fast,  short  length  messages  are  delivered 

faster  through  alternate  routes  since  the  direct  ones  are  heavily 

* 

used  by  longer  length  messages  using  special  routing. 

This  Is  clearly  evident  In  Fig.  2.Ac  where  the  traffic  is  high. 

The  best  performance  occurs  at  the  lowest  value  of  DELTA  and  update 
of  20. 

Update  of  1*0  confirms  the  above  observation  for  every  traffic 
load  (see  Fig.  2 .4a  ,b,c).  In  Flg.2.1|cv  DELTA  ■ 70  and  UP  =>  20  show 
worse  performance  than  that  of  UP  * *»0  for  the  same  value  of  DELTA. 

At  this  point,  due  to  the  high  traffic,  heavy  update  data  traffic 
Is  further  degrading  the  network's  performance. 

As  the  update  gets  slower,  the  queue  lengths  change  faster  than 
the  update  can  sense.  Then,  best  performance  moves  along  the  Inter- 
mediate to  high  values  of  DELTA.  This  occurs  because  the  delay  table 
information  gets  out  of  date,  the  wrong  decision  Is  taken,  and  thus 
the  queues  are  longer.  This  is  more  evident  with  high  traffic  loads, 
(e.g.,  Fig.Z.te,  with  update  of  100). 

When  the  update  gets  very  slow,  high  values  of  DELTA  give  better 
performance,  it  Is  Interesting  though  to  note  that  for  DELTA  ■ 70 
and  UP  * 200  in  Flg.2Aa  and  ike,  there  is  an  increase  of  the  ratio  value. 

it  can  be  explained  two  ways: 

1.  There  Is  some  randomness  Introduced  In  the  decision  making 
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process  due  to  the  loss  of  effectiveness  of  the  slow  update.  Then 
this  randomness  results  in  the  lowest  ratio  value  for  DELTAS  between 
1*0  and  50  instead  of  higher  ones  near  70. 

2.  It  might  be  thought  of  as  the  traffic's  peculiarity  (a 
sudden  pile  up  of  messages)  at  the  time  the  sample  was  taken  which 
results  In  better  performance  when  A0<  DELTA  < 50. 

If  we  Increase  the  threshhold  value  to  35,  then  the  Immediate 
effect  Is  a higher  ratio  value  (see  Fig. 25a),  This  shows  up  more 
dramatically  as  the  traffic  gets  higher  (see  Fig. 25b),  In  Fig.  2.6a 
and  In  Fig. 2.6b,  the  threshhold  value  is  increased  even  more  and  the 
same  observation  Is  true. 

Even  with  reduced  special  routing  when  the  traffic  Is  low  and 
the  update  fast,  the  system's  performance  Is  comparable  to  that  of 
a low  threshhold  (see  Fig. 2.5^  Fig. 25a).  As  the  traffic  gets  higher 
and  the  update  slower,  then  the  ratio  value  gets  high,  as  expected 
(see  Fig.  2,5b,  Fig.  2,6b). 

In  Fig.  27a  and  Fig.  27b,  the  relation  of  the  ratio  value  and 
DELTA  Is  examined  as  the  amount  of  special  routing  varies  at  a fixed 
update  value  of  1*0  for  two  traffic  loads  respectively. 

Again,  the  benefit  of  the  special  routing  Is  evident  In  both 
figures. 

For  low  traffic,  moderate  bias  towards  the  direct  route  shows 
best  results.  At  a higher  traffic,  long  messages  (TRSLD  * 70)  are 
doing  better  when  strongly  biased  towards  the  direct  route.  The 
same  thing  holds  even  for  TRSLD  * 35.  At  a TRSLD  * 50,  there  Is  no 
significant  change  In  performance  with  a changing  DELTA.  When  special 
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routing  Is  higher,  (TRSLD  ■ 12,  20)  low  bias  values  perform  better. 

For  the  purpose  of  a better  Illustration  of  the  special 
routing  effect,  F1g.2J}  was  plotted.  Two  extreme  threshhold  values 
were  used  at  a fixed  update  of  20  and  three  traffic  loads.  The  high 
traffic  curves  show  better  the  need  for  special  routing. 

When  the  update  Is  decreased  (see  Fig.  2.9^  2.9k),  Interesting  facts 
showed  up  in  the  high  traffic  plot.  Best  performance  for  high  DELTA 
values  almost  everywhere.  When  the  amount  of  special  routing  Is 
changed,  either  a lot  of  special  routing  (TRSLD  ■ 12,  20)  will  compen- 
sate for  the  slow  update  or  very  little  (TRSLD  ■ 70). 

Bias  toward  the  direct  route  for  moderate  length  messages 
(TRSLD  * 35,  50)  causes  long  delays  In  the  direct  routes  which 
results  In  a higher  ratio  value.  This  seems  to  suggest  that  In 
high  traffic  and  slow  update.  It  Is  better  to  directly  route  only 
very  long  length  messages.  Those,  together  with  the  ones  directly 
routed  by  mistake  because  of  out-of-date  Information,  result  In  a 
better  system  performance  than  If,  In  addition,  moderate  length 
messages  block  the  direct  routes.  That  Is,  the  use  of  alternate 
routing  under  the  above  stated  conditions  results  In  better  system 
performance. 

The  above  assertions  are  being  supported  from  traffic  data. 

The  number  of  messages  delivered  which  followed  other  than  the  direct 
path  and  took  1 or  2 or  3 or  ...  >,  10  "excess  hops"  to  arrive  at 
their  destination  was  measured  for  DELTA  ■ 70  and  two  threshhold  values 
of  50  and  70.  From  Fig. 2. 1 2,  although  fewer  messages  followed  the 
direct  route  In  the  case  of  TRSLD  ■ 70  than  in  the  case  of  TRSLD  ■ 50, 
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the  resulting  actual  delay  per  message  for  TRSLD  - 70  Is  less  than  that 
at  TRSLD  = 50.  Therefore,  In  TRSLD  = 70  case,  alternate  routes  are 
beneficial.  In  TRSLD  = 50  case,  overuse  of  direct  routes  degrades 
system  performance.  This  Is  because  when  most  messages  take  a direct 
route,  subsequent  long  messages  have  to  take  an  Indirect  one.  The 
result  Is  a high  average  delay.  When  small  and  moderate  length  messages 
are  following  alternate  routes,  then  direct  routes  are  used  only  for 
long  messages.  This  results  In  a better  overall  average  message  delay. 

Finally,  the  performance  of  the  bias  factor  delta  Is  investigated 
as  a function  of  the  update  period  when  very  little  special  routing  takes 
place  (TRSLD  **  50)  for  two  traffic  loads  In  Fig.  2.10a  and  in  Fig.  2.10b,  For 
low  traffic  and  fast  update,  low  or  moderate  bias  towards  the  direct 
routes  results  in  the  same  ratio  value,  while  strong  bias  does  worse. 

For  slow  update,  strong  encouragement  towards  the  direct  route  performs 
better  (see  Fig. 2.10a,)  For  higher  traffic.  Fig. 2.10b  strong  bias  towards 
a direct  path  always  performs  better. 

The  effectiveness  of  DELTA  to  prevent  looping  has  been  investigated 
when  the  update  Is  slow  (UP  = l 00)  and  the  special  routing  Is  moderate 
(TRSLD  = 20)  for  two  traffic  loads.  In  Fig, 2.11  two  tables  are  presented 
which  show  the  number  of  messages  delivered  which  followed  other  than 
the  direct  route  and  took  I or  2 or  3 ...  or  i,  10  "excess  hops"  to 
arrive  at  their  destination.  Two  values  of  DELTA  10  and  70  are 
recorded.  They  show  that  under  the  above  stated  conditions  a high  bias 
factor  value  prevents  looping.  This  shows  up  best  for  high  traffic 
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condl t ions. 
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Summary 

Low  Traffic 

Best  performance  when  the  update  period  Is  smaller  than  the 
average  message  length,  at  low  values  of  the  asynchronous  update  and 
a value  of  the  bias  factor  DELTA  approximately  twice  the  average 
message  length.  The  best  asynchronous  update  threshhold  value  Is  12, 

When  the  update  period  gets  greater  than  the  average  message 
length,  a low  threshhold  value  of  12  still  maintains  best  performance 
at  values  of  DELTA  approximately  twice  the  average  message  length. 

If  the  threshhold  value  Is  high  so  that  only  very  long  messages  are 
specially  routed,  then  the  fastest  update  performs  better. 

In  the  worst  case  of  slow  update  and  high  threshhold  value,  much 
higher  than  the  average  message  length  values  of  the  bias  factor 
DELTA  give  better  results  (DELTA  ■ 70). 

High  Traffic 

Best  performance  at  update  period  smaller  than  the  average 
message  length,  threshhold  value  of  12  and  DELTA  value  of  approx- 
imately equal  to  the  average  message  length. 

For  a slow  update,  a low  threshhold  value  of  12  maintains  best 
performance  at  the  highest  DELTA  value  used  (DELTA  ■ 70). 

In  the  worst  case  of  slow  update  and  high  threshhold  value,  a 
DELTA  value  of  50  results  In  better  performance. 
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Figure  2.<*{a)  Ratio  vs.  Delta  for  TRSLD  = 12,  1/y  = 1.14. 
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Ratio  vs.  Delta  for  TRSLD  **  12,  1/y  = 
UP  = 20,  UP  * 40,  UP  - 100,  UP  * 200. 
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Figure  2.5(b)  Ratio  vs.  Delta  for  TRSLD  - 35,  1/Y  * 
UP  = 20,  UP  « bO,  UP  = 100,  UP  *=  200. 
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Figure  2.6(a)  Ratio  vs.  Delta  for  TRSLD  ■ 50,  1/y  * 1.1**  and 
UP  = 20,  IIP  = 40,  UP  » 100,  UP  = 200. 
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F i gi re  2.7(b)  Ratio  vs.  Delta  for  UP  - 1*0,  \/y  «*  .97  and 

TR$LD=1 2,  TRSLD-20,  TRSLD-35,  TRSLD-50,  TRSLD-70. 
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Figure  2.9(a)  Ratio  vs.  Delta  for  UP  ■ 100,  1/Y  - 1.14,  and 

TRSLD-12,  TRSLD-20,  TRSLD-35,  TRSLD=50,  TRSLD=70. 
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Ratio  vs.  Delta  for  UP  «*  100,  ]/y  *1.14  and 
TRSLD-12,  TRSLD-20,  TRSLD-35,  TRSLD-50,  TRSLD-70. 
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EXCESS  HOPS 

NUMBER  OF  MESSAGES  DELIVERED 
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1365. 

1367. 


UP  - 100 
TRSLD  - 20 


EXCESS  HOPS  NUMBER  OF  MESSAGES  DELIVERED 

DELTA  - 10  DELTA  = 70 


1 

258 

225 

2 

177 

122 

3 

78 

29 

4 

0 

0 

5 

29 

8 

6 

29 

8 

7 

15 

4 

8 

12 

6 

9 

6 

3 

2 10 

15 

7 

The  total 
The  total 


number  of  messages  del lvered  for  DELTA 
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10  Is  1543. 
70  Is  1 584. 


Figure  2.11  Excess  hops  vs.  number  of  messages  delivered 
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70  is 

1507. 

figure  2.12  Excess  hops  vs.  number  of  messages  delivered 
for  TRSLD  - 10,  70. 


CHAPTER  3 


DETERMINISTIC  ROUTING  STRATEGY 


Introduction 

In  a deterministic  routing  strategy,  a predetermined  function  of 
all  incoming  messages  is  sent  to  each  output  queue. 

The  optimal  routing  problem  consists  of  finding  the  deterministic 
routing  strategy  (or,  equivalently  the  set  of  routes)  according  to  which 
messages  have  to  be  routed  so  as  to  optimize  a well  defined  performance 
function. 

Under  appropriate  assumptions,  the  optimal  routing  problem  can  be 
formulated  as  a nonlinear  multicommodity  flow  problem  [FRAN  71  A]. 

Several  techniques  for  solving  the  multicommodity  problem  can  be 
found  in  the  mathematical  programming  literature  [DANT  63]  ,[GERL  73]: 
however,  the  straighi -forward  application  of  these  techniques,  which  are 
appropriate  for  the  solution  of  very  general  multicommodity  problems,  to 
the  routing  problem  in  store-and-forward  nets  are  computationally  cumber- 
some. Therefore,  an  extremely  fast  routing  technique  has  to  be  used.  For 
that  reason,  considerable  effort  has  been  spent  In  developing  heuristic 
and  suboptimal  methods  [FRAN  71®*  FULT  72].  The  exact  mathematical  problem 
has  been  attacked  by  [GERL  73^]-  He  formulated  and  solved  it  as  an  uncon- 
strained nonlinear  programming  problem.  In  fact,  he  presents  two  methods: 
the  flow  deviation  (FD)  method  and  the  external  flows  methods. 
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To  make  the  presentation  of  this  report  more  complete,  the  FD 
method  will  be  adapted  here  for  the  solution  of  the  deterministic 
problem.  It  is  chosen  among  others  because  of  Its  simplicity  and  good 
performance.  The  deterministic  case  for  the  same  eight  node  network, 
as  in  Chapter  2,  is  simulated  with  the  computer  In  order  to  evaluate 
the  adaptive  routing  algorithm. 

In  section  3.1,  the  definition  of  the  routing  problem  Is 
formulated. 

In  section  3.2,  the  multicommodity  flow  problem  Is  presented  In 
terms  of  the  total  flow  In  every  link,  along  with  the  constraints  that 
it  satisfies. 

In  section  3.3,  the  performance  function  chosen  is  described. 

In  section  3«*»,  a brief  description  of  the  solution  of  the  problem 
is  presented  according  to  the  FD  method  and  an  outline  of  the  FD 
algorithm  is  formulated. 

In  section  3.5,  the  application  of  the  FD  method  to  the  routing 
problem  is  investigated. 

In  section  3.6,  the  algorithm  Is  applied  to  the  routing  of  an 
eight  node  network. 

In  section  3.7,  an  evaluation  of  the  adaptive  routing  algorithm 
is  presented  by  comparing  its  performance  with  the  performance  of  the 
deterministic  routing. 


Consider  a network  consisting  of  nodes  (I  = I,  ...,N)  and 
links  L.  (j  = i , . . . ,M) , In  such  a network,  we  are  required  to  route 
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a quantity  r.^  of  type  (1,j)  messages  from  M(  (source  node)  to 
(destination  node),  for  all  pairs  (i,j).  The  routing  problem  is  defined 
as  the  problem  of  finding  the  routing  of  ail  such  messages  which 
optimizes  a well-defined  performance  function  such  that  a set  of 
constraints  are  satisfied. 


Definition:  We  define  f. 


(M) 


to  be  the  average  flow  [bits/sec]  produced  in 


link  i by  messages  traveling  from  source  node  k to  destination  node  & . Let 
f.  be  the  total  average  flow  in  link  i,  given  by 

f.  A Z I f>*> 

' k=l  *=1  1 

where  N is  the  number  of  nodes  in  the  net. 

Definition:  We  denote  by  "commodity  (i,j)M  the  messages  which  are  generated 
in  node  i and  have  destination  node  j.  Commodity  (i,j)  produces  the  single 
commodity  flow  f1,1  ,J  . All  commodities  together  produce  the  multi  commod  i ty 
flow  f. 


Definition:  Let  [messages/sec]  be  the  required  average  rate  of  trans- 

mission of  messages  from  source  i to  destination  j,  and  let  1/y  [bits/ 
message]  be  the  average  message  length.  We  define  the  requirement  r.^ 
[bits/sec]  as  follows: 
r. . = Y. ./ u 

ij  U/y 

and  define  the  requi rement  matrix 
Ri  <rij>- 

The  routing  problem  then  can  be  in  general  formulated  in  the  following 


way : 
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given:  a network  of  N nodes  and  H links  and  a N x N matrix  R ■*  [r{  J, 

the  requirement  matrix,  whose  entries  are  non-negative. 

minimize:  P(f) 

over  f ~ . 

where  f * (f.,f  ,...,f  ) and  P(*)  Is  a well  defined 
**  ' L H 

performance  function. 

constraints:  (a)  f is  a multicommodity  flow  satisfying  the  require- 

ment matrix  R. 

(b)  f must  satisfy  some  additional  constraints  (e.g. 

capacity  constraints  on  each  link  or  cost  constraints) 
In  the  case  of  a store-and- forward  communication  network,  the  routing 
problem  has  the  following  formulation: 

given:  a network  of  N nodes  and  M links  and  a N x N matrix  R » [r.  J, 
whose  entries  are  non-negative. 


. . . t(k) 

minimize:  T = 

1L  W 

\ 

~ (t.)k 

1/K 

1 

f . 

1 

over  f 

LM 

Y • 

Y 

o 

i 

-h 

1 

constraints:  (a)  f is  a multicommodity  flow  satisfying  the 

requirement  matrix  R. 

(b)  f 1 C. 

where  C = (Cj ,C2>. . • »CM) 

3.2  Representation  of  a multicommodity  flow 

A practical  representation  of  a multtcomodlty  flow  Is  through  the 
vector  f of  the  flow  of  the  links. 

f = ^f  1 » f 2 * * * * ,fM^ 
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where  f.  is  the  total  flow  on  link  (or  channel)  1,  sum  of  all  commodities 
flowing  through  i. 

f does  not  completely  characterize  a multicommodity  flow:  for  in- 
stance, two  different  sets  of  routes  might  result  In  the  same  f.  In  many 
multicommodity  problems,  however,  the  performance  function  depends  only 
on  the  total  flows  f . . In  such  cases,  the  f - representation  is  sufficient. 
The  problem  under  investigation  belongs  to  such  a category.  It  is  of 
interest,  therefore,  to  investigate  the  properties  of  the  set  F defined 
as  the  set  of  all  feasible  f. 

F = F H F. 

-a  ~b 

where  F = { f | f satisfies  multicommodity  constraint  (a)} 

Ffa  “ satisfies  capacity  constraint  (b)} 


3.2.1.  The  multicommodity  constraint  (a) 

Let  f.j^m,n^  be  the  flow  on  link  (i,j)  due  to  commodity  (m,n).  For 
the  conservation  of  flow  at  each  node,  for  each  commodity  (l,j),  and 
non-negativity  of  the  flow  in  each  link,  the  following  constraint  is  true. 


f-  r 


Z f..(m’n)  - Z f..(m’n)  .[ 

j-1  J‘  J-l  U 


. r 
^ mn 


I = 1,2 N 

For  non-negativi ty  of  the  flows  we  have: 
(m,n) 


if  I * m 
if  i f1  m , n 
if  i “ n 


(3-0 


f . 


>-  o V i ,j 


(3.2) 
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Let  f 


(m,n)  A 


(m,n)  (m,n) 

1 1 • r2 


j ^ where 


f^(m,n)  js  the  flow  due  to  commodity  (m,n)  on  the  link  labeled  1,  and 
let  F^m,n^  = {f (m,n) j ^ (m,n)  satjsfy  equation  (3.1)  and  (3.2)}.  Then 
p(m,n),  defined  by  linear  equations  (3.1)  and  inequalities  (3.2)  is  a 
convex  polyhedron.  By  definition,  the  total  flow  f Is  given  by 


N N 

f = l 1 

m=l  n=1 

It  can  be  shown  [HU  69]  that,  when,  for  each  commodity  (l,J), 
f'1*"’  spans  the  respective  feasible  set  F'  then  the  total  flow  f 

according  to  (3.3)  also  spans  a convex  polyhedron. 

Therefore,  F is  a convex  polyhedron  (Fig.  3.1).  The  flows  corres- 

**3 

ponding  to  the  flows  of  the  "corners"  (extreme  points)  of  the  polyhedron 
F have  an  interesting  property:  they  are  the  shortest  route  flows' 

“*3 

[FORD  62].  It  is  also  shown  that  all  shortest  route  flows  correspond  to 

extreme  points  of  F . 

~a 


;(m,n) 


(3-3) 


Figure  3.1  Feasible  set  F and  extremal  flows. 

—a 

'a  shortest  route  flow  Is  a multicommodity  flow  whose  routes 
can  be  described  by  a shortest  route  matrix,  computed  for  an  arbitrary 
assignment  of  lengths  to  the  links. 
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3.2.2  The  capacity  constraint  (b) 


The  set  F , defined  by: 


lb  " {f If  * c} 


is  a convex  set. 


Hence  the  feasible  set  F ■ F A F.  is  also  a convex  set.  See  Fig.  3.2. 


Figure  3.2  feasible  set  F ■ F n F, . 

~ —a  ~b 


3.3  The  Performance  Function 


As  a performance  function,  the  Mean-kth-Power  Delay  T(K)  |s  USed. 


M \ ~I 

z T (Ti)K 

i-1  Y i 


i a<® 


(3.1.) 


where 


> PC  - A. 

i i 


and  A,  ■ ^f t , 


The  function  (3.1*)  is  a strictly  convex  function  [ME i S 71], 

For  K ■ 1 , we  obtain  the  average  delay  as  in  eq.  (1.7).  For  K ■ 2, 
the  weighted  variance  of  message  delay  Is  minimized  [ME I S 71],  If  K 
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is  allowed  to  become  large,  optimization  under  this  performance 
function  leads  to  equalizing  the  message  flow  in  the  links. 

The  Mean-kth-Power  Delay  does  not  have  a physical  significance. 

It  is  more  sensitive  to  the  needs  of  the  individual  node. 

3.1*  Descr  l£l>  on  of  the  Flow  Deviation  Method 

Before  we  go  into  the  description  of  the  FD  method,  an  important 
observation  about  the  behavior  of  the  performance  function  will  facil- 
itate the  solution  of  the  problem. 

A simple  Inspection  of  the  problem  presented  in  section  l*.l  shows 

that 

£im  T (f ) = + oo 

fi  *Cf 

This  fact  will  prevent  f from  approaching  the  boundary  of  F. 

- ~ b 

when  T is  minimized. 

Such  behavior  is  quite  general  for  store-and-forward  networks. 

When  the  capacity  constraints  are  satisfied  with  equality,  usually 
some  saturation  occurs,  the  queues  at  nodes  grow  large,  and  the 
delay  T increases  rapidly.  We  can  then  say  that  the  performance 
function  incorporates  the  capacity  constraints  as  a penalty  function. 

This  property  is  very  important  because  it  guarantees  automatically 
the  (b)-feasibi 1 ity  during  the  application  of  the  usual  nonlinear 
optimization  techniques,  once  a feasible  flow  f®  is  formed. 

Therefore,  if  we  have  a starting  feasible  flow  f^,  we  can  dis- 


regard the  capacity  constraints,  and  we  can  consider  the  routing 
problem  as  an  unconstraint  multicommodity  flow  problem.  Then  the 


above  problem  is  classified  as  an  unconstrained  multicommodity  flow 
problem  with  nonlinear  performance  function. 

The  problem  Is  separable  convex  and  differentiable.  The  approach 

for  solving  it  consists  of  approximating  the  performance  function  with 

the  tangent  hyperplane,  which  is  expressed  In  terms  of  the  partial 

derlvitlves  {0P/0f.}.  Then  the  min  cost  problem  of  the  linearized 

AA 

problem  is  the  shortest  route  flow  , where  the  length  of  link  I Is 
defined  as  0P/0f..  Such  shortest  route  flow  represents  the  direction 
of  the  steepest  descent  flow  deviation. 

The  above  idea  Is  the  essence  of  the  FD  method,  which  consists  of 
repeated  evaluations  of  steepest  descent  directions  and  of  one  variable 
minimization  along  such  directions;  the  method  is  conceptually  very 
similar  to  the  gradient  method  applied  to  nonlinear  minimization 
problems. 

The  idea  of  using  shortest  routes  (computed  with  partial 
derivitives)  is  not  new.  Dafermors  [DAFE  69]  solved  various  traffic 
problems,  formulated  as  unconstrained,  convex  m.c.  flow  problems. 

Yaged  [YAGE  71]  solved  the  min  cost  capacity  assignment  problem  for 
a communication  network,  which  was  formulated  as  an  unconstrained 
m.c.  flow  problem. 

*A  separable  m.c.  flow  problem  has  the  form 
M 

P(f)  = E P,(f.) 

M ' 

i A 

A shortest  route  flow  is  a multicommodity  flow  whose  routes  can 
be  described  by  a shortest  route  matrix,  computed  for  an  arbitrary 
assignment  of  lengths  to  the  links. 
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3«i».  1 The  FD  Method 

Gerla  [GERL  73 B]  proved  that  If  f Isa  stationary  flow*,  then  the 
shortest  route  flow  evaluated  under  metric  £„  m 0P/0f„  represents  the 
flow  deviation  of  steepest  decrease  for  P.  This  fact  suggests  a method 
which  Is  called  Flow  Deviation  Method,  for  the  determination  of 
stationary  solutions  of  the  unconstrained  nonlinear  convex  differ- 
entiable flow  problems. 

Let's  define  FD  as  an  operator 

FD(v,A)0  f = ( l - X) f + Av  - f' 

where 

v is  a properly  chosen  m.c.  flow  eFa 

*N#  *%* 

A is  the  step  size  (0  C A c 1) 

Now  for  each  feFa,  we  want  to  determine  a pair  (v,A)  In  such  a way 
that  the  repeated  application  of  FD(v,  A)  produces  a sequence  {fn} 
which  converges  to  a stationary  flow. 

'ff  A 

For  a function  P(f)  which  Is  continuous  nondegenerate  and 
lower  bounded,  the  following  conditions  are  sufficient  for  the  con- 
vergence of  the  FD  to  a stationary  flow: 

*f  is  defined  as  stationary  If  for  any  infinitesimal  pertur- 
bation“*6f  (such  that  f + 6f  Is  also  a m.c.  flow) 

P(f  + <5f)  i P(f) 

*-*  -W  <s» 

A local  minimum  Is  always  stationary.  The  opposite  Is  not  true:  for 
instance,  a local  maximum  Is  also  stationary. 

P(f)  is  defined  t^>  be  nogdegenerate,  for  any  two  distinct 
stationary  flows,  say  f and  f , we  have 

P(f')  * P(f2) 

«<• 
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(0  4P(f)  >0  fef 

-w  ■*» 

(li)  ^P(f)  = 0 -*■  f stationary 
where 

4P(f)  = P(f)  - P(FD<S>f) 

>*  **  «*• 

Conditions  (i)  and  (ii)  require  that  the  FD  method  be  a true  steepest 
descent  method. 

Under  the  assumptions  tht  P(f)  is  continuous,  lower  bounded, 

first  partial  derivitives  continuous  and  nonnegative;  second  partial 

derivitives  < 00 , P(f)  nondegenerate,  the  following  definition  of 
•** 

FD(v,X)  satisfies  conditions  (I)  anc  (5*1; 

v = shortest  route  flow  under  metric  2 
*•  K 

A = rrinimizer  of  P[(i-A)f  + Av] , 0 i A £ I 

where  U.  = ^ is  the  assigned  weight  (length)  to  the  links  of 
K d tK 

the  network.  The  shortest  route  rr..  is,  therefore,  the  route  for 

ij 

which  E 2 is  minimum. 

Kor..  K 
•J 

An  aigcrithm  then  based  on  the  definition  of  the  FD  operator  is 
outlined  as  follows: 

0 

1.  Find  a feasible  starting  flow  f 

2.  Let  n * 0 

3.  Find  the  shortest  route  flow  v under  metric  € 

•**  i\ 

1.  Find  A such  that  mirP((l-A)fn  + Xvn] 

OiAsl 
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b.  It  (P(fn)  - P(fn+*)}  < e stop  where  c Is  an  acceptable 
positive  tolerance.  Otherwise,  let  n ■ n+1  and  go  to  3. 

The  algorithm  converges  to  stationary  points;  however,  the 
stationary  points  of  stable  equilibrium  are  the  local  minima; 
therefore,  we  can  assume  that  the  algorithm  converges  to  local 
minima.  In  the  case  of  P(f)  strictly  convex,  the  algorithm  clearly 
converges  to  the  global  minimum. 

In  order  to  find  a feasible  starting  flow,  i.e.,  f^eF,  several 
methods  are  available  [CANT  72],  [GERL  73B].  The  one  that  Is  used 
here  consists  of  picking  any  feF  , and  then  reducing  the  flows  In  all 

~ <v3 

links  by  a scaling  factor  ft  until  a feasible  flow  f®  = pfeF  is 

<v 

0 

obtained,  f satisfies  a reduced  requirement  matrix  “PR.  The 

0 

FO  method  is  applied  using  f as  the  starting  flow  and  R as  the 

•*»  u 

starting  requirement.  After  each  FD  iteration,  the  value  of  pis 
Increased  up  to  a levei  very  close  to  saturation.  The  search  for  a 
feasible  flow  is  terminated  when  one  of  the  two  following  cases 
occurs:  either  Pi  1 and  a feasible  flow  Is  found;  or  the  network  is 
saturated,  T (f ) is  minimized  and  p < l.  In  the  iatter  case,  the 
problem  is  infeasible  (with  respect  to  a given  tolerance). 

The  FD  algorithm  then  consists  of  two  phases,  Phase  i and 
Phase  2 (see  Appendix  ll).  In  Phase  i a feasible  flow  f^  is  found 
(if  it  exists),  or  the  problem  is  declared  infeasible.  In  Phase  2 
the  optimal  routing  is  obtained. 

3.5  The  Routing  Problem 

In  the  routing  problem  formulated  in  section  3.i»  the  performance 
(K) 

function  Tv  7 (f ) , equation  (3.1*),  is  continuous,  lower  bounded  by  zero, 

A* 
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strictly  convex  (separable  sum  of  strictly  convex  functions)  and 

differentiable.  The  feasible  set  F = F OF.  is  a convex  polyhedron. 

~ -a  ~o 

Therefore,  if  the  problem  is  feasible,  there  is  a unique  stationary 

point,  which  is  the  global  minimum.  The  capacity  constraints  are 

included  in  T'  (f)  as  penalties.  Then  if  we  can  find  a feasible 
**# 

0 

starting  flow  f eF,  the  problem  of  section  3.1  can  be  regarded  as  an 
unconstrained  m.c.  flow  problem  and  be  solved  with  the  FD  method. 

(k) 

Let  us  check  if  Tv  '(f)  satisfies  the  conditions  for  con- 

( k)  k 

vergence.  it  is  equivalent  to  check  [Tv  '(f)]  instead. 


(K)  K M a.  K M l-K 
[T(K)(f)]K  = £ ~ (T,)K  = I 


i 


1-1  Y 


i = l 


(Ci"fi) 


The  first  partial  derivitive  is 


3f, 


[ 


(c.-f.) 


K+l 


for  this  function  to  be  positive  we  require  that  K+i  is  an  even 

JU 

number.  Then  K is  a finite  odd  number.  The  second  partial  derivite  is 
0 i t j 


a(T(K))K 


l-K  2KC.  + f.(K-K) 

H r 1 ] < 

Y (c.-f.)^2 


When  t lie  first  partial  derivitive  is  positive,  then  the  shortest 
route  algorithm  is  guaranteed  to  work  since  the  weights 

,*(t(k>ik 

v — ) of  the  links  are  positive. 

ar  • 
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Therefore  the  sufficient  conditions  or  the  first  two  derivitives 
on  the  performance  function  are  satisfied.  The  FD  algorithm  converges 
to  a global  minimum. 

3.6  Applications:  Deterministic  Routing  of  the  AUTODiN  Network 

The  appiication  of  the  FD  problem  was  investigated  for  the  8-node 
European  AUTODIN  network.  Fig.  2.3.  The  communication  link  capacity 
is  equal  for  every  link  and  equal  to  2400  bits/sec.  The  traffic 
requirement  is  assumed  Poisson  starting  from  a minimum  of  65  bits/sec 
and  increased  until  saturation.  The  saturation  occured  at  about  <>76 
bit/sec.  Several  values  of  K were  investigated.  In  Fig.  3.4  the 
average  delay  per  message  is  plotted  versus  the  input  requirement. 

The  values  of  K were  taken  K = J,3,5»M»  After  K ■*  5 the  results 
were  practically  the  same. 

It  is  of  interest  that  when  K is  increased,  the  flow  in  the 
links  tends  to  be  spread  uniformly  to  ail  links  which  results  in 
lower  average  message  delay,  it  is  also  true  that  the  more  equalized 
the  fiow  in  the  links  the  fastest  the  network  on  the  average  goes 
into  saturation.  See  Fig.  3.4. 

3.7  Evaluation  of  the  Adaptive  Routing  Algorithm 

The  performance  of  the  adaptive  routing  algorithm  was  evaluated 
by  comparing  it  with  the  performance  of  the  deterministic  routing 
algorithm. 

Adaptive  routing  is  applied  to  operational  networks  while 


l. 


deterministic  routing  is  used  in  the  design  of  networks  according 
to  their  input  requirements.  From  simulation  data,  it  has  been 


s 
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observed  that  In  an  operational  network  It  Is  Important  that  the 

adaptive  routing  algorithm  distributes  the  traffic  to  the  link  queues 

in  such  a way  as  to  keep  them  short  and  well  balanced.  This  results 

in  a reduced  average  message  delay.  In  the  deterministic  routing  case, 

the  above  mentioned  effect  Is  observed  as  we  Increase  the  value  of  K 

(K) 

in  the  performance  function  T'  . For  this  reason,  the  comparison 
of  the  two  algorithms  Is  done  for  K **  II  (deterministic  case)  as  this 
represents  the  case  closest  to  the  function  of  the  adaptive  algorithm. 
When  1C  > l 1 the  resulting  average  message  delay  obtained  was 
practically  the  same  as  for  K = 11.  Therefore,  for  the  network  under 
consideration  and  its  input  requirements,  K = 11  is  considered  the 
highest  value  that  K takes. 

In  Fig.  3.5  the  average  message  delay  was  plotted  versus  the  Input 
requirements  for  both  the  deterministic  and  adaptive  routing  algorithms. 
In  every  case,  the  input  requirements  were  assumed  to  follow  a Poisson 
distribution  with  parameter  equal  to  the  Input  requirement.  In  the 
adaptive  routing  for  each  different  Input  requirement  the  factors  of 
DELTA  and  TRSLD  were  chosen  to  be  those  which  resulted  In  the  least 
average  message  delay.  These  values  were  DELTA  *=  10  and  TRSLD  = 12. 

The  adaptive  algorithm  simulationwas  run  for  one  half  hour  simulation 
time  in  every  case. 

From  the  plotted  results  we  observe  three  regions  of  Interest: 

Light,  med iurrv  and  heavy  input  traffic  requirements. 

It  is  generally  expected  that  the  deterministic  routing  should 
result  in  a lower  average  message  delay  than  the  adaptive  routing 
since  it  i s an  optimal  method.  The  adaptive  routing  Is  suboptlmal  and 
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in  addition  the  update  information  data,  which  do  not  exist  in  the 
deterministic  routing,  further  increase  the  average  message  delay. 

In  light  input  traffic  conditions,  the  delay  In  the  adaptive 
case  is  higher,  as  expected,  because  even  If  a few  routing  decisions 
were  not  optimal,  they  affected  significantly  the  total  average  delay 
per  message  since  there  are  relatively  a few  messages  In  the  system 
to  begin  with. 

In  medium  input  traffic  conditions,  that  most  networks  usually 
operate  upon,  it  Is  encouraging  to  observe  that  adaptive  routing 
resulted  in  a lower  average  message  delay.  The  adaptive  routing 
decisions  are  near  optimum  in  this  case.  In  the  deterministic  routing 
case,  vie  essentially  look  at  the  traffic  situation  of  the  network  at 
a single  point  in  time  and  find  the  mathematically  precise  optimal 
routing.  In  the  adaptive  routing  case  we  observe  the  average  message 
delay  for  a period  of  time.  Then  on  the  average  the  adaptive  routing 
shows  a better  performance  over  a long  period  of  time,  for  the  same 
input  traffic  conditions. 

For  heavy  input  requirements,  the  system  was  driven  into 
saturation  faster  with  the  adaptive  routing  than  in  the  deter- 
ministic. The  average  message  delay  increases  exponentially,  in 
this  case,  the  nonopt imai i ty  of  the  adaptive  decisions  is  pronounced. 
Generally  heavy  traffic  conditions  drive  operational  networls  into 
saturation  very  fast. 
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CHAPTER  U 

PARTITIONING  OF  A NETWORK 


Introduction 

Given  a genera  I,  d I str i buted  topology  store-and-forward  communi- 
cation network,  it  is  generally  recognized  that  It  should  be  parti- 
tioned into  smaller  components  for  a better  efficiency  of  the  solu- 
tion of  the  routing  problem.  This  is  also  true  for  design  consid- 
erations [GERL  73B]. 

As  far  as  the  deterministic  routing  Is  concerned,  the  algorithm 
presented  by  Geria  requires  a computational  time  proportional  to 

O 

approximately  fr  (where  N is  the  number  of  nodes  in  the  network)  and 

2 

a storage  requirement  of  about  N . It  seems,  therefore,  to  be 
inadequate  to  handle  problems  with  N > I00„ 

In  the  adaptive  routing  case,  a direct  application  of  the  algorithm 
to  a large  network  (hundreds  of  nodes)  would  require  large  storage  for 
every  node  for  storage  of  deiay  tables,  routing  tables,  etc.  A large 
amount  of  computation  time  Is  required  to  maintain  those  tables.  The 
requ i rerr.cnt  for  exchanging  long  update  information  between  nodes 
would  reduce  the  effective  link  capacities.  Excessive  time  delays 
can  occur  in  the  percolation  of  the  data  Information  which  would 
result  in  an  out-of-date  information  when  it  reaches  its  destination. 
Final i>,  as  a result  of  high  storage  and  computational  requirements 
placed  upon  every  node,  nodes  will  become  more  expensive  to  maintain. 
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On  solution  to  such  a problem  could  be  to  partition  the  original 
network  into  subnetworks  for  which  the  above  mentioned  algorithms  are 
still  applicable  and  for  which  an  economy  of  storage  and  computation 
may  be  achieved. 

The  idea  of  partitioning  was  suggested  by  Gerla  [GERL  73 B] . A 
lot  of  clustering  techniques  are  found  in  the  pattern  recognition 
literature.  Those  techniques  can  apply  to  network  problems  when  the 
objects  to  be  partitioned  can  be  represented  by  a network  graph  [FU  75] 
This  idea  is  used  in  the  proposed  algorithm,  in  the  large  store-and- 
forward  network  design  problem  [KAMO  76]  the  network  is  partitioned 
in  tree  structured  subnetworks. 

In  this  chapter  a partitioning  algorithm  is  developed  mostly 
suited  to  existing  communications  network  topology. 

In  section  4.1  a definition  of  the  partition  used  is  stated, 
along  with  general  observations  about  the  traffic  requirements  of  a 
partitioned  network.  Considerations  for  a partitioning  approach  are 
discussed  and  two  partitioning  solutions  are  proposed. 

In  section  4,2  the  hierarchical  method  approach  is  presented. 

In  section  4.3  a clustering  method  is  developed.  An  algorithm 
is  proposed  for  the  partitioning  of  communications  networks  of  the 
distributed  type.  The  main  idea  in  the  aigorithm  is  the  use  of  a 
labeling  technique  to  discover  the  appropriate  regions  for  partitioning 
Two  examples  are  worked  out  at  the  end  of  the  section  which  illustrate 
the  application  of  the  aigorithm. 


According  to  such  a definition,  we  can  associate  with  each  cluster 
Its  own  subnetwork,  and  study  It  Independently  from  the  rest  of  the 
network.  The  definition  also  Implies  that  each  subnetwork  must  be 
connected.  Every  subnetwork  carries  internal  traffic  (generated,  and 
with  destination,  inside  the  cluster)  as  well  as  external  traffic 
(traffic  from  Internal  to  external  nodes  and  vice  versa). 

When  partitioning  a large  network,  consideration  to  the  following 
important  questions  must  be  given  [GERL.  73B], 

1.  How  can  nodes  be  grouped  so  that  the  network  can  be  partloned 
In  such  a way  that  the  number  of  nodes  within  each  partition 
can  be  specified. 
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2.  The  message  traffic  of  the  different  partitions  should  be 
fairiy  balanced. 

3.  The  geographical  diameter  of  each  partition  should  be 
considerably  smaller  than  the  average  distance  between 
different  partitions. 

k.  For  design  considerations  the  geographical  diameter  of 
each  partition  should  be  selected  according  to  the  line 
cost  characteristics. 

Two  approaches  are  being  proposed  for  the  solution  of  the  partitioning 
problem. 

l.  Hierarchical  approach 
2.  Clustering  approach 

k.Z  Hierarchical  Method  Approach 

A hierarchical  network  is  one  in  which  message  are  selected 

according  to  a routing  policy  which  is  influenced  by  the  specific  net- 
work topology  [FULT  72].  Special  topological  features  associated  with 
general  networks  can  be  exploited  in  order  to  simplify  and  increase 
the  operational  efficiency  of  the  routing  procedures. 

The  hierarchical  structure  can  be  broken  into  a number  of 
hierarchical  levels.  The  application  of  this  principle  permits  a 
proper  separation  so  that  each  level  can  be  implemented  essentially 
independently  and,  therefore,  can  be  modified  and  extended  without 
affecting  the  other  levels.  In  many  cases,  this  form  of  partitioning 
is  natural  due  to  the  geographical  locations  of  the  nodes. 

The  hierarchical  structures  proposed  assume  that  the  network  can 
be  partitioned  usually  on  a geographical  basis,  into  local,  regional, 
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and  national  subnetworks,  in  order  to  make  the  discussion  easier,  only 
networks  with  regional  and  national  partitions  are  examined. 

The  hierarchical  structure  utilizing  this  concept  of  partitioning 
Is  described  as  follows:  The  network  Is  divided  into  subnetworks 
according  to  the  partitioning  definition.  Nodes  in  each  partition  are 
assigned  either  central  or  local  nodes.  Within  any  subnetwork  any 
connection  is  possible.  Connections  between  subnetworks  are  only 
allowed  between  central  nodes  of  each  region.  Any  connection  between 
subnets  is  possible.  Fig.  4.1  shows  an  example  of  such  a network 
topology  resulting  from  this  imposed  hierarchical  structure  where  each 
subnet  contains  only  one  central  node. 

Each  node  can  be  labeled  by  the  two-tople  (R,J)  where  R Is  the 
subnet  number  and  J is  the  node  number  within  the  subnet.  At  each 
central  node,  a list  must  be  maintained  of  the  other  central  nodes 
reachable  from  it.  This  list  of  reachable  subnets  from  a given  subnet 
can  be  transmitted  to  all  nodes  In  the  given  subnet. 


Subnet  I Subnet  2 


Figure  4.1  A simple  hierarchical  network. 
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Unfortunately,  In  topological  structures  with  only  one  central 
node  per  subnet,  the  failure  of  a central  node  can  disconnect  a local 
node  in  the  subnet  from  the  remainder  of  the  subnets  in  the  network. 
Then  possibly  two  nodes  (or  more)  must  be  declared  as  central  nodes. 
At  this  point,  nice  properties  of  the  structure  In  Fig.  A. 1 start  to 
disappear.  Fig.  k.2  shows  a possible  network  topology. 


Subnet  I Subnet  2 


Figure  U.2  A two  central  node  per  subnet 
hierarchical  network. 


If  the  central  node  within  each  subnet  Is  forced  to  be  directly 
connected  (dashed  lines)  then  routing  in  the  national  and  regional 
subnetworks  can  be  achieved  independently.  However , if  a message 
must  be  routed  from  a given  subnet  to  another  subnet,  there  is  a 
choice  of  routing  the  message  to  either  of  the  central  nodes. 

Since  the  regional  networks  would  not  have  information  as  to 
which  central  node  Is  the  best  choice,  larger  message  delays  can 
occur.  For  example.  If  a message  must  go  from  subnet  k to  subnet  1 
and  the  i Ink  between  the  two  central  nodes  has  failed,  then  much 
larger  message  delay  results  If  It  enters  the  national  network  from 

I J 
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the  right  hand  node  than  if  it  enters  by  the  left  hand  side  control 
node. 

In  addition,  a failure  of  the  direct  link  between  central  nodes 
would  force  messages  destined  from  one  subnet  to  another  to  go  through 
local  nodes  which  would  result  in  additional  delay  and  more  complicated 
routing  strategy. 

In  order  to  assess  the  difficulties  at  the  hierarchical  networks 
described,  their  topoiogles  must  be  studied  In  the  network  design 
problem. 

4.3  Clustering  Method  Approach 

A graph  theory  approach 

The  network  is  represented  as  a connected  graph. 

Definition:  A node  cut-set  of  a connected  graph  G = (V,T)  Is  a non- 
empty set  of  nodes  whose  removal  (including  all  links  incident  to  nodes 
in  the  set)  from  G results  in  a disconnected  or  trivial  subgraph.  An 
S-T  node  cut-set  is  a nonempty  set  of  nodes  (not  containing  nodes  in 
S and  T)  whose  removal  from  G interrupts  all  paths  between  two  subsets 
of  nodes  S and  T. 

A link  cut-set  of  a connected  graph  can  be  defined  similarly. 

A node  cut-set  is  also  called  an  articulation  set.  "Node  cut- 
set" and  "articulation  set"  will  be  used  interchangeably. 

From  observation  of  distributed  network  graphs,  it  is  evident 
that  as  a consequence  of  poor  connectedness,  there  exist  articulation 
sets  that  would  help  partition  the  network. 

Then  the  problem  is  to  detect  smal 1 articulation  sets  which  would 
reveal  reasonably  large  subnets  in  a parent  net.  No  restriction  is 
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made  to  finding  only  one  single  minimal  articulation  set  of  a large 
net,  since  ail  reasonably  large  subnets  are  desirable. 

Efficient  methods  for  finding  minimal  node  cut-sets  or  link  cut- 
sets are  avaiiabie  in  [FORD  56],  [FULK  62]  but  they  don't  solve  the 
general  problem. 

An  algorithm  containing  a generalized  version  of  Frisch's 
algorithm  [FRIS  67]f  [Fu  75],  has  been  developed.  It  sequentially 
results  in  subnets  from  the  original  net.  The  conditions  for 
desirable  size  subnets  can  be  specified  by  a splitting  criterion. 

Some  definitions  should  be  given  at  this  point  relevant  to  the 
connectivity  of  a network.  Some  important  theorems  are  also  stated. 

Definition:  The  connectivity  of  a node  In  a graph  G Is  the  number  of 
links  incident  to  the  node.  The  minimum  connectivity  6(G)  of  G Is 
the  smallest  of  the  connectivities  of  nodes  in  G. 

Definition:  A node  is  star  connected  when  It  is  connected  to  at 
least  one  node  of  connectivity  one. 

Definition:  The  connection  number  cj(G)  (or  the  node  connectivity) 
of  a graph  G is  the  number  of  nodes  In  a minimal  node  cut-set.  The 
cohesion  number  yjG)  of  a graph  G Is  the  number  of  links  in  a minimal 
edge  cut-set. 

In  a graph,  the  connection  number  Ui(G),  the  cohesion  number  x(G) 
and  the  minimum  connectivity  6(G)  are  related  by  an  inequality  due  to 
Whitney  [WHIT  32]. 
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Theorem <1.3.1 : (Whitney).  For  any  graph  U,  the  following 
inequal l ty  holds: 

w(G)  £ x(G)  * 6(G) 

Definition:  if  h is  an  integer  £ i,  G is  said  to  be  h-connected  if 

w(G)  2 h,  and  G Is  said  to  be  h-coherent  if  x(^)  2 h. 

Theorem If  a graph  G is  h-connected  or  h-coherent,  where  h 
is  some  positive  integer,  then  the  connectivity  of  every  node  in  the 
graph  G must  be  greater  than  or  equal  to  h. 

For  the  proof  of  theorems ^ . 1 and  4.3.2  refer  to  [WHIT  32]. 

Theorem43>3 < The  necessary  and  sufficient  condition  for  a graph 
G «=  (V,T)  to  be  h-connected  is  that  two  arbitrary  distinct  nodes  V.  and 

i 

V,  v.,  v.cV,  can  be  joined  by  h paths  which  have  no  nodes  (links)  in 
common  except  the  nodes  v.  and  v. . The  proof  of  the  theorem  is  based 
on  Ford  and  Fulkerson's  labeling  algorithm.  For  a detailed  proof, 
refer  to  [BERG  62]. 

The  last  theorem  does  not  give  a direct  tool  to  verify  a graph 
being  h-connected.  To  show  that  a graph  consisting  of  N nodes  is 
h-connected,  where  h is  an  integer,  1 i h < H,  one  has  to  check  ail 
( ) combinations  cf  pairs  of  nodes.  V/hen  N becomes  large,  this 
approach  is  obviously  Impractical. 

A different  approach  is  developed  using  the  h-connectedness 
concept.  In  order  to  gain  an  understanding  as  to  how  this  concept  is 
used,  three  examples  will  be  presented  demonstrating  three  different 
situations.  The  size  of  the  networks  in  the  examples  are  kept  small 
for  demonstration  purposes. 
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In  Fig.  4.3(a)  a network  (represented  as  a graph)  contains  ten 
nodes,  each  of  which  Is  connected  to  exactly  three  other  nodes.  The 
network  Is  1-connected  since  the  minimal  node  cut-set  contains  one 
node.  The  articulation  set  here  Is  the  minimal  one.  Removal  of  node 
a or  b wl 1 1 separate  the  net  Into  two  subnets. 

If  a network  has  a high  value  of  h-connectedness,  It  Is  tightly 
connected.  However,  If  It  has  a low  value  of  h-connectedness.  It  Is  not 
always  loosely  connected.  For  example,  In  Fig.  4.3(b),  the  network  Is 
1-connected  even  though  It  is  rather  strongly  connected,  because  the 
connectivity  of  the  network  Is  spoiled  !>v  a few  nodes  of  connectivity  1. 

In  Fig.  4.3(c)  a large  network  Is  at  most  ^-connected,  by  Theorem  4.3.2 
because  the  connectivity  of  the  network  Is  4.  There  exists  a node  cut- 
set containing  5 nodes  (u,  v,  w,  x,  y).  The  size  of  the  node  cut-set 
exceeds  the  connectivity  of  the  network,  but  It  Is  very  small  compared 
with  the  size  of  the  network. 

The  removal  of  this  node  cut-set  breaks  the  parent  network  Into 
two  subnetworks  of  almost  equal  size.  Then  It  Is  necessary  to  find  an 
S-T  node  cut-set  between  two  certain  subsets  of  nodes  S and  T whose 
sizes  are  larger  than  the  maximum  degree  of  the  network.  In  this 
case,  the  minimal  node  cut-set  Is  not  meaningful  because  It  falls  to 
reveal  the  interesting  subnetworks. 

4.3.1  Partitioning  of  a Network 

The  subnetworks  hidden  In  a large  parent  network  are  required 
to  satisfy  the  following  splitting  criterion: 

(1)  The  sizes  of  the  articulation  sets  should  be  as  small  as 


possible 


A 
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(2)  The  sizes  of  the  resulting  subnets  should  be  reasonably 
large  and  connected  In  order  to  be  subnetworks. 

Any  trivial  graph  will  not  be  considered  as  a subnet. 

In  the  case  where  an  articulation  set  is  equal  to  an  s-t  minimal 
node  cut-set  between  two  nodes  s and  t,  by  the  well-known  max-flow, 
mln-cut  theorem  [FORD  62],  the  articulation  set  can  be  found  by  maxi- 
mizing the  flow  from  s to  t with  the  constraints  presented  In  Chapter  3 
on  a flow  f(Vj,Vj)  which  are  associated  with  the  flow  conservation  and 
1 ink  capacities. 

A label ing  algorithm  Is  a method  used  to  find  the  maximum  flow  and 
also  find  the  minimal  s-t  node  cut-set.  Frlsh's  labeling  algorithm  is 
one  of  the  most  efficient  methods. 

As  already  mentioned,  there  are  cases  where  the  size  of  a delsl rable 
articulation  set  is  larger  than  any  minimal  s-t  node  cut-set  between  any 
two  nodes  s and  t.  In  such  cases,  a flow  problem  of  single  source  and 
destination  should  be  generalized  to  a flow  problem  of  multiple  sources 
and  destinations.  For  example.  In  Fig.  4.3(c),  the  articulation  set 
(u,  v,  w,  x,  y}  can  be  detected  by  considering  a flow  problem  from  a 
set  of  multiple  sources,  S « (a,  b,  c,  d,  e,  f)  to  a set  of  multiple 
destinations;  T ■ (g,  h,  I,  j,  k,  1 }.  A flow  problem  of  multiple 
sources  and  destinations  can  be  reduced  to  a flow  problem  of  single 
source  and  destination  In  the  following  way:  Replace  the  set  of 
multiple  sources  (Including  removal  of  all  links  incident  to  the  set) 
by  an  artificial  source,  and  connect  the  new  source  to  ail  nodes 
adjacent  to  the  set  of  multiple  sources.  Then  treat  the  set  of 
mutliple  destinations  in  the  same  way. 
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An  algorithm  Is  proposed  here,  containing  a generalized  version  of 
Frisch's  labeling  algorithm  [CHEN  73].  The  generalized  labeling  algor- 
ithm (GFLA)  Is  given  in  Appendix  111.  The  algorithm  consists  of  the 
fol lowing  steps. 

1.  Specify  a splitting  criterion;  namely,  specify  the  reference 
size  m of  a desirable  subnet,  and  the  reference  size  k. 

Where  k £ 2.  represents  the  allowable  deviation  from  m. 

Go  to  step  2. 

2.  From  the  graph  consisting  of  the  set  of  nodes  under  inves- 
tigation, choose  two  small  connected  subgraphs  which  are 
about  as  far  as  possible  from  each  other.  Consider  the  two 
sets  of  nodes  In  these  two  subgraphs  as  the  multiple  sources, 
S,  and  the  multiple  destinations,  T.  Go  to  step  3. 

3.  Apply  the  GFLA  to  the  set  of  nodes  under  Investigation. 

Obtain  the  set  of  labelled  and  uniabelled  nodes,  and  the 
articulation  set,  denoted  as  L,  U,  A.  Let  | L|  , |u  | and  | A| 
be  their  sizes.  List  L,  U,  A,  | L| , |u|.  Go  to  step  4. 

i».  Examine  If  there  are  any  star  connected  nodes  t A.  If 
there  are,  let  this  set  be  M.  When  n^eM  is  also  mj=s^:S 
create  N ■ tn^n^M,  n^ffs).  Store  N and  define  L ■ L-N, 

I L|  - I U - I N|  and  U = U + N,  | u|  - | u|  + [ N|  . List  L,  U, 

| L|  , | u|  . Go  to  step  5.  If  there  are  not,  go  to  step  5. 

5.  Break  the  set  of  nodes  under  investigation  into  two  sets 
Land  U by  removing  the  set  of  links  connecting  a labelled 
and  an  uniabelled  node.  Examine 
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(a)  If  | L | £■  m+k  go  to  step  7. 

If  | L ( > m+k  store  L.  L needs  further  Inves- 
tigation. Go  to  step  2. 

(b)  If  ]U|  * m+k  go  to  step  7. 

If  |u | > m+k  store  U.  U needs  further  inves- 
tigation. Go  to  step  6. 

6.  Examine  N.  I f N **  0 then  let  N c_  S or  N c_  T.  Find  a 
smail  connected  subgraph  T or  S about  as  far  as  possible 
from  S or  T.  Go  to  step  3.  If  N « 0 go  to  step  2. 

7.  List  L or  U.  These  subgraphs  are  the  legitimate  subnets. 

Go  to  step  8. 

8.  Combine  each  subgraph  of  size  less  than  m-k  with  one  of  Its 
neighboring  subgraphs  In  such  a way  as  to  produce  a new 
possible  subnet.  Go  to  step  9. 

9.  Stop. 

A flowchart  of  the  algorithm  ts  given  In  Fig.  b.b. 

The  algorithm  given  above  will  yield  sequentially  subnets  in  a 
finite  number  of  steps.  It  preserves  ail  nodes  In  the  original  net- 
work and  aii  links  between  nodes  within  the  same  subnet. 

The  GFLAwill  find  all  distfnct++  flow  paths  between  the  source 
nodes  and  the  terminal  nodes,  and  will  terminate  when  the  terminal 
nodes  cannot  be  labelled.  At  the  termination  of  the  GFLA  the  set 
of  labelled  nodes  L and  the  set  of  unlabelled  nodes  U are  found. 

L is  not  empty  since  S Is  labelled.  U Is  not  empty  since  T is  unlabelled. 

Two  paths  are  distinct  when  they  have  no  node  or  link 
In  common. 
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Store  L.  L needs  further  Investigation , 


List  L,  j L j • This  subgraph  Is  a 
legitimate  subcluster 


If  U i,  k + m 


Store  U.  U needs  further  investigation. 


Jf  N 4 * 


Let  N c S or  N c T. 
Find  a”~small  subgraph 
T or  S as  far  as 
possible  from  S or  T. 


List  U,  |Uj.  This  subgraph  is  a 
legitimate  subciuster. 


any  listed  | L | or 


Combine  each  subgraph  of  size 
less  than  m - k with  one  of  Its 
neighboring  subgraphs  in  such  a 
way  as  to  produce  a new  possible 
subnet. 


Figure  k. A (Continued)  Flowchart  of  the  partitioning  algorithm, 


Then  the  articulation  set  Is  the  set  of  labelled  nodes  which  are 
connected  to  an  unlabelled  node  by  a link,  l.e. 

A = {x:  (x,y)  e (L,U)  }. 

The  labelling  In  the  GFLA  Is  a one  end  labelling.  It  uses  both 
plus  and  minus  labels.  The  scanning  of  the  nodes  Is  done  on  a first- 
labelled  first-scanned  basis.  It  also  applies  directly  to  the  original 
graph  (network). 

4,3.2  Remarks  on  the  Algorithm 

The  articulation  set  Is  not  unique.  The  subnets  that  the  algorithm 
will  define  depend  on  the  cholr.c  of  the  set  of  source  and  the  set  of 
terminal  nodes  every  time  the  I*  applied. 

Usually  nodes  w)  tr  hi<  c-.rnu^.  v Ity  make  a good  choice  of 
source  nodes.  Aft.  - .e  nodes  are  chosen,  a good  choice  for  ter- 
minal nodes  Is  stll'  ’qh  connected  nodes  as  far  as  possible  from  the 
source  nodes.  Star  connected  nodes  are  usually  high  connected  also. 

Then  star  connected  nodes  fulfilling  the  above  requirements  are  satis- 
factory selection  for  source  at  terminal  nodes. 

The  number  of  nodes  In  S or  T will  depend  on  the  size  of  the 
network  to  be  partitioned  and  Its  topological  structure. 

For  large  size  networks  usually  more  source  or  terminal  nodes 
are  required  for  fast  partitioning  to  begin  with. 

For  small  site  networks  the  minimal  criterion  works  well,  l.e. 
only  one  source  and  only  one  terminal  node.  In  this  case,  the  minimal 
articulation  set  will  represent  the  strength  of  the  weakest  part  of 
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Example  i : The  network  shown  In  Fig.  4.5  consists  of  31  nodes.  Let 
the  splitting  criterion  be  m “ 10  and  k *•  3.  Choose  as  the  set  of 
source  nodes  S ■ (3,8)  and  the  set  of  terminal  nodes  T ■ (30,30. 

After  the  application  of  the  GFLA  four  distinct  paths  are  found.  They 
are  indicated  by  an  arrow  (Fig.  **.6 ),  pointing  In  the  direction  of  the 
flow,  next  to  every  link  in  the  path. 

The  set  of  iabeiied  nodes  are  shown  carrying  their  label  (s)  next 
to  their  node  number.  The  flow  paths,  the  labelled  and  unlabeiled 
nodes  are  shown  in  Fig.  4.6. 

The  articulation  set  is  A * (3, **,7}.  The  set  of  iabeiied  nodes  is 
L = (1 ,2, 3, 4, 5, 6, 7, 8).  The  set  of  unlabeiled  nodes  consists  of  the 
rest  of  the  nodes  in  the  original  network,  | L|  * 8,  |l)|  * 23. 

The  links  connecting  the  nodes  eA  with  an  unlabeiled  node  are 
broken  and  the  resulting  graph  Is  shown  In  Fig.  4.7,  |l|  = 8 < m+k. 
Therefore,  the  set  L constitutes  a legitimate  subnet.  |li|  ■»  23  > m+k 
therefore  U needs  further  investigation. 

Examine  U. 

Choose  as  the  new  set  of  source  nodes  S ■ (9,11)  and  T = (19,31). 
After  the  application  of  the  GFLA.  two  distinct  paths  are  found.  The 
flow  paths  the  set  L and  U are  shown  in  Fig.  4.7.  Aj  * (15,18), 

L,  = (9,10,11,12,13,14,15,16,17,18),  U,  - (19,20,21,22,23,24,25,26,27, 
28,25,30,31  )f 1 1 j | = 10,  |Uj|  = 13.  After  the  removal  of  the  links 
converting  the  articulation  nodes  with  an  unlabeiled  node,  the  graph 
in  Fig.  4.8  resui ts. 

|L, I = 10  < m+k  therefore,  Lj  is  a legitimate  subnet.  |Uj|  ■ 

13  “ m+k  therefore  Uj  Is  a legitimate  subnet.  The  algorithm  has 
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Figure  **.5  A 30-node  network. 
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yield  three  connected  subnets  from  the  original  network.  The  size 
of  all  subnets  Is  greater  than  m-k  and  less  than  m+k.  Therefore,  the 
algorithm  stops. 

Example  2:  The  network  shown  In  Fig.  4.  9 consists  of  95  nodes.  Let 
the  splitting  criterion  be  m ■ 12  and  k ■ 6.  Choose  as  the  set  S - 
{4,7}  and  as  T ■ {89,77,94}.  After  the  application  of  the  GFLA  four 
distinct  paths  are  found.  The  flow  paths,  the  sets  L,U,  are  shown  In 
Fig.  4.10.  The  articulation  set  A ■ {17,30,70}.  The  set  L ■ {1,2, 3, 4, 5, 
6,7,8,9,10,11,12,13,14,15,17,18,19,20,21,22,23,24,25,26,27,28,30,32,70}. 
The  set  U consists  of  the  rest  of  the  nodes  In  the  network.  |l|  - 30, 

| U | ■ 65.  All  the  nodes  In  the  articulation  set  A are  star  nodes  then 
L - L - A - {1,2,3,4,5,6,7,8,9,10.11,12,13,14,15,18,19,20,21,22,23,24, 
25,26,27,28,32}  and  U consists  of  the  rest  of  the  nodes  In  the  original 
network.  |l|  - 27,  |U | ■ 68.  Break  the  set  of  links  connecting  the 
labelled  nodes  with  an  unlabelled  one.  The  resulting  graph  shows  In 
Fig.  4.11.  |L|  ■ 27  > iTH-k  therefore  L needs  further  Investigation. 

| U | ■ 68  > nH-k  therefore  U needs  further  Investigation. 

Examine  the  set  L first. 

Let  the  source  nodes  for  the  subgraph  L be  Sj  ■ {15}  and  T^  ■ {26}. 
After  the  application  of  the  GFLA  one  distinct  path  Is  found,  shown 
In  Fig.  4.10.  The  articulation  set  Aj  “ {4},  Lj  ■ {4,6,7,8,9,10,11, 
12,13,14,15,18,19,20,21,22,23},  U,  - {1,2,3,5,24,25,26,27,28},  | L, | - 
17.  |U,|  ■ 9.  The  node  In  Aj  Is  a star  connected  node  therefore  M ■ 

N ■ {4}.  Redefine  Lj  ■ L^  - {4},  Uj  ■ Uj  + {4}.  Then  |Lj|  ■ 16, 

|U,|  - 10. 


k. 
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Break  the  set  of  links  connecting  the  nodes  In  Lj  with  an 
unlabelled  node.  The  resulting  subgraph  shows  In  Fig.  4.12.  ||_j  | ■ 

17  < m+k.  Therefore,  Is  a legitimate  subnet.  |Uj  | - 10  < rrH-k. 
Therefore,  Is  a legitimate  subnet. 

Examine  the  subgraph  whose  nodes  belong  In  U. 

Let  S2  “ A * {17,30,70}  and  T2  “ {62,94}.  After  the  application 


of  the  GFLA,  three  distinct  paths  are  found.  A2  " {30,55,78}.  Then 
M = {30,78}  and  N = {78}.  The  set  of  labelled  nodes  L2  **  {16,17,29,30, 

31,33,34,  35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55, 
70,71,78,79,80,81,82,83,84,85,86,87,88,89,90,  91,92,93},  the  set  U2  - 
{56,57,58,59,60,61,62,63,64,65,66,67,68,69.72,73.74,75,76,77,94,95}. 

|L2I  - 46,  |U2|  = 22.  Since  N f 0,  redefine  Lj-Lj-N-Lj-  {78} 
and  U2  “ U2  + N - U2  + {78}.  Then  |L2|  - 45,  |U2|  *=  23.  The  flow 
paths  found,  the  labelled  and  unlc'^’led  nodes  are  shown  in  Fig.  4.12. 

Break  the  links  connecting  the  set  of  labelled  nodes  with  an 
unlabelled  node.  The  resulting  subgraph  Is  shown  In  Fig.  4.13. 

|L2|  *-  45  > m+k.  Therefore  L,  needs  further  Investigation, 
i U2 1 = 23  > ftH-k.  Therefore  U2  needs  further  Investigation. 

Examine  L2  first.  Choose  = {17,30}  and  T^  = {89}.  After  the 
application  of  the  GFLA  two  distinct  paths  are  found.  A^  •*  {30,55}, 

M - {30},  N - 0,  L3  «=  {16,17,29,30,31,33,34,35,36,45,46,47,48,49,55}. 

L3  = {37,38,39,40,41 ,42,43,44,52,53,54,70,71 ,79,80,81 ,82,83,84,85,86,87, 
88,89,90,91,92,93}.  ILj)  *=  15,  | | » 28.  The  flow  paths  found,  the 
set  of  labelled  and  unlabelled  nodes  are  shown  In  Fig.  4.13. 

Break  the  links  connecting  labelled  and  unlabelled  nodes.  The 
resulting  subgraph  shown  In  Fig.  4.|4. 


|Lj|  ■ 15  < m+k,  therefore  L^  Is  a legitimate  subnet.  |U^|  “ 

28  > m+k,  therefore  needs  further  investigation. 

Examine  U,.  Let  ■ {58}  and  ■ N • {78}.  After  the  application 
of  the  GFLA,  two  distinct  paths  are  found.  A^  ■ {61,  62}  M ■ N ■ {62}, 

- N - {56,57,58,60,61,95}.  + N - {62, 63, 64, 65, 66, 67, 68, 

69,72,73,74,75,76,77,78,94}.  |lJ  - 7,  |uj  - 16.  The  flow  paths 
and  the  sets  L^,  show  In  Fig.  4.14. 

After  breaking  the  links  connecting  a labelled  and  an  unlabelled 
node,  the  resulting  subgraph  Is  shown  in  Fig.  4.15. 

|Lj  ■ 7 < m+k,  therefore  is  a legitimate  subnet.  |U^|  ■ 

16  < m+k,  therefore  Is  a legitimate  subnet. 

Examine  U^.  Choose  $,.  ■ {70},  T^  ■ {89}.  After  the  application 
of  the  GFLA,  two  distinct  paths  are  found.  The  flow  paths  and  the  set 
of  labelled  and  unlabelled  nodes  are  shown  In  Fig.  4.15.  A,.  ■ {70,71}, 

M - {70},  N -t.  L5  - {37,38,31,40,41,42,43,44,52,53,54,70,71,79,80, 

82,83},  V$  - {81,84,85,86,87,88,89,90,91,92,93},  |L$|  - 17,  | | - II. 

After  breaking  the  links  connecting  labelled  and  unlabeiled  nodes, 
the  resulting  subgraph  Is  shown  In  Fig.  4.16. 

|Lj.|  « 17  < m+k,  therefore  L^  is  a legitimate  subnet.  |u^|  • 

11  < m+k,  therefore  Is  a legitimate  subnet. 

The  original  network  Is  partitioned  sequentially  In  seven  subnetworks. 
The  size  of  these  subnets  is  less  than  m+k  and  greater  than  m-k.  There- 
fore, the  algorithm  stops. 

The  partitioned  network  Is  shown  in  Fig.  4.16. 


26(27,.) 
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Figure  4. 10  A 95-node  network  showing  tne  first  step  towards  partitioning. 
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CHAPTER  5 

ADAPTIVE  ROUTING  FOR  LARGE  STORE-AND-FORWARD  COMMUNICATIONS  NETWORKS 
Introduction 

In  chapter  2 an  adaptive  routing  algorithm  was  developed.  The 
algorithm,  as  discussed  in  chapter  4,  is  not  directly  applicable  to 
large  s to re-and- forward  networks.  If  the  network  Is  properly 
partitioned,  then  the  algorithm  can  be  applied  after  certain  necessary 
modifications  are  made. 

In  this  chapter,  the  adaptive  routing  algorithm  is  modified  to 
adjust  to  a partitioned  network.  The  necessary  equations  are  also 
given  such  that  the  deterministic  algorithm,  adapted  In  chapter  2, 
can  be  applied  to  a large  computer  communication  network. 

In  section  5.i  a solution  to  the  deterministic  routing  problem 
for  large  networks  is  given.  The  adaptive  routing  problem  is 
discussed  in  relation  to  the  partitioning  approach. 

In  section  5.2  considerations  for  large  networks  are  presented. 
Intuitive  arguments  are  given  about  the  traffic  requirements  and 
queueing  delays  in  a large  store-and-forward  system.  The  adaptive 
routing  algorithm  is  divided  according  to  the  partitioned  network 
into  a subnet  algorithm  and  a master  net  algorithm. 

In  section  5.3  the  subnet  routing  algorithm  is  presented. 

In  5.4  the  master  net  routing  algorithm  Is  given. 
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In  section  5.5  an  example  Is  simulated  with  the  computer.  The 
simulation  language  used  Is  discussed. 

In  section  5.6  the  simulated  system  characteristics  are  given, 
and  the  conclusions  drawn  from  the  simulation. 


5.1  Deterministic  routing  for  large  store-and-forward  networks 


According  to  the  partitioning  algorithm,  we  can  partition  a net- 
work into  subnetworks.  Then  study  each  subnet  separately.  We  can  then 
associate  with  each  subnet  a supernode  and  study  the  master  network  that 
connects  all  supernodes.  In  such  a study,  we  only  consider  the  external 
traffic,  le.  ,the  traffic  between  nodes  belonging  to  different  subnets. 

Therefore,  If  m and  n are  supernodes,  we  define  the  composite  requirement 

.(0)  „ 


r'  as  follows: 
mn 


for  m )*  n 


UN<">  JeN<">  r'J  ' 
0 , for  m » n 


The  total  average  delay  T can  be  expressed  In  terms  of  the  delays 
of  master  net  and  subnets  in  the  following  way: 

P (k) 

T-  Z - T(k)  (5.D 

k=0  r 

where:  T(0)  = delay  of  master  net 

T(k),  k - 1 p - delay  of  subnets 


r<0)  t J l r<0) 

i m mn 

m“l  n*l 


thruput  of  master  net 


no 


* Z » r[k^  ■ thruput  of  subnet  k 
i,jeN<k)  fJ 

N N 

r = Z Z r..  ■ total  thruput. 
i«l  j=l  '•* 

The  simple  relation  (5.1)  was  obtained  using  Little's  result 
as  shown  in  [GERL  73A] . The  delay  T(k)  has  an  interesting  physical 
interpretation:  T(0)  is  the  average  delay  in  the  internode  communications, 
and  T(k)  is  the  average  delay  suffered  by  Internal  and  transit  traffic 
in  subnet  k. 

The  deterministic  problem  for  a large  network  can  be  formulated 
in  a similar  way  as  in  chapter  2. 

After  partitioning  the  network  into  subnets  each  subnet  is  con- 
sidered separately.  The  existing  algorithms  for  deterministic  routing 
are  directly  applicable.  The  average  delay  per  message  T(k)  for  each 
subnet  k is  exactly  calculated  using  the  flow  deviation  algorithm 
presented  in  chapter  2. 

The  master  net  can  be  viewed  as  a subnet  Itself  since  the  size  of 
the  number  of  nodes,  is  small.  Then  the  average  delay  of  the  master 
net  T(0)  can  be  calculated  in  the  same  way.  The  total  average  delay 
T of  the  entire  network  is  given  by  equation  5.1. 

Therefore,  by  partitioning  the  original  network  into  subnetworks, 
the  solution  of  the  deterministic  routing  problem  is  possible. 

The  adaptive  routing  problem  can  also  be  solved  by  using  the 
partitioning  approach. 

The  algorithm  presented  in  chapter  2 can  be  modified  into  a 


subnet  routing  strategy  and  a master  net  routing  strategy 
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In  each  subnet  a number  of  nodes  are  chosen  as  central  nodes. 
The  exact  number  of  control  nodes  wilt  depend  on  the  size  of  the 
subnet.  It  is  assumed  that  the  central  nodes  are  connected  directly 
and  thus  form  the  master  net. 

5. 2 Large  network  adaptive  routing  considerations 

The  problems  discussed  below  are  present  for  small  networks  as 
well.  But  they  are  more  imminent  and  magnified  when  large  networks 
are  considered.  In  the  algorithm  presented  in  chapter  2,  the  delay 
table  information  transmitted  at  update  is  a "snapshot"  of  the 
traffic  situation  in  the  network  at  the  Instant  of  the  update.  If 
the  variations  in  queue  lengths  are  too  "fast"  the  situation  may 
change  within  the  update  period. 

In  general,  we  have  a "sampt ing"  situation.  The  queues  have 
variations:  random  due  to  the  stochastic  nature  of  the  inputs; 
periodic  due  to  the  time-of-day  variations,  seasonal  variation,  etc. 
We  are  attempting  to  adapt  to  these  changes.  The  sampling  theorem 
applies  in  which  case  we  have  to  sample  "fast  enough"  to  be  able 
to  sense  {"see"  ) the  variations. 

The  stochastic  variations  are  related  to  message  length  and 
message  arrival  times.  Their  interactions  have  been  assumed  to  be 
Independent  {according  to  Kleinrock's  independence  assumption). 

But  in  actuality,  since  the  message  length  is  kept  constant  while  it 
is  routed  from  its  source  to  its  destination,  there  is  a dependence 
and  interaction  of  these  variations.  How  they  interact  exactly  is 
very  hard  to  analyse.  Unless  we  sample  those  variations  fast 
enough,  we  have  no  way  to  do  a sensible  adaption  to  these  changes. 
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Very  fast  update  periods  would  be,  in  general,  undesirable  because  of 
the  additional  load  they  put  on  the  data  Information  transfer. 

Then  the  following  modifications  in  the  adaptive  algorithm  wi i i 
be  considered  to  smooth  the  above  mentioned  problems. 

"Smoothing":  In  the  case  of  stochastic  variations,  there  are  two 

distinguishable  problems.  First,  the  problem  of  "aging,"  i.e.,  the 
choice  of  a path  through  a distant  node  is  based  on  data  that  is 
several  update  periods  old.  Given  a large  enough  network,  "aging" 
will  affect  optimality  no  matter  how  short  the  update  period  is. 

Second,  the  data  sent  at  update  may  be  "unrepresentative"  In 
some  fashion  and  the  correction  of  the  delay  tables  information  made 
accordingly  may  be  very  nonoptimai  over  the  update  period.  That 
could  cause  a "deciding"  node  to  misjudge  even  an  adjacent  node. 

The  approach  taken  here,  to  try  to  correct  the  above  problems, 
is  to  use  the  statistics  of  the  queues  in  the  network. 

In  the  adaptive  algorithm  of  chapter  2,  the  queue  length  at  the 
instant  of  the  update  Is  incorporated  directly  in  the  delay  tables 
information.  When  the  update  of  information  Is  not  extremely  fast, 
the  instantaneous  queue  length  may  change  within  the  update  period. 

In  other  words,  one  could  expect  that  the  instantaneous  queue  length 
is  not  representative  of  the  situation.  Thus,  we  attempt  to  bring 
it  closer  to  the  mean  value  of  the  queue.  The  weighted  sum  of  the 
instantaneous  queue  length  and  the  mean  value  of  the  queue  length  Is 
considered  replacing  the  instantaneous  queue  length  in  the  delay 
table  calculations. 

From  simulation  data,  it  has  been  observed  that  th£re  are 
"reliable"  and  "unreliable"  routes  for  a message  to  follow,  i.e.. 
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routes  consisting  of  links  where  the  queue  length  changes  more 
slowly'bn  the  average"  and  others  where  queue  lengths  fluctuate  all 
over  the  place.  The  goal  is  to  try  to  follow  reliable  routes  as 
much  as  possible.  In  an  unreliable  route,  large  changes  of  the  queue 
lengths  will  take  place  before  the  message  routed  to  a destination 
node  reach  the  node  and  thus  take  circuituous  paths.  The  mechanism 
used  to  penalize  the  links  whose  variance  is  large  is  a penalty  term, 
which  results  as  a bias  against  choosing  these  links.  The  biasing  is 
done  by  multiplying  the  queue  length  times  the  variance  of  the  queue 
length.  The  above  term  Is  incorporated  in  the  delay  tables  calcu- 
lations. The  adaptive  algorithm  is  modified  according  to  the  above 
arguments  and  divided  into  a subnet  routing  strategy  and  a master 
net  routing  strategy. 


5.3  Subnet  routing  strategy 

Each  node  keeps  local  routing  tables  where  destinations  within 

the  subnet  are  listed.  In  addition,  the  node  receives  a master  net 

minimum  delay  vector  from  each  local  central  node  to  any  other  subnet. 

Let  Sj  be  the  symbol  of  subnet  S where  node  1 belongs.  And  let 

j ^ be  the  index  of  the  central  node  j local  to  subnet  S.. 
i 

The  master  net  minimum  delay  vector  Is  then  given  by 


MT.  (Sn)  = min  (min  MT.  (i  ,Lm)) 
i Lm  JS  5D 
SD  S 


(5.2) 


where  MT.  (i  ,Ln)  is  the  delay  table  entry  of  the  master  net  that 

JS  SD 
s 

represents  the  estimated  delay  to  go  from  the  initial  node  , lja 

s 


m 


central  node  j local  to  the  subnet  S where  the  message  source  s belongs, 

to  node  1.  , i.e.,  central  node  1 local  to  the  subnet  S where  the 
bD 

message  destination  node  D belongs,  through  the  outgoing  link  Lm  of 

initial  node  j.  . MT.  (S  ) then  is  the  estimated  minimum  delay  to 

s J S D 
s 

go  from  to  destination  subnet  S^. 

The  delay  table  entries  for  messages  with  destination  within  the 
subnet  are  calculated  via  the  delay  table  equation 

TK(D,Lg)  - [a(K,Lg)  + MVQ(K.Lg)]  x 1/2  + T(D)  + DELTA  (5.3) 

where  Q(K,Lg)  Is  the  Instantaneous  queue  length  of  node  K on  link  Lg, 
MVQ(K,Lg)  is  the  mean  value  of  the  same  queue.  T(D)  is  the  delay 
vector  entry  for  destination  D as  defined  In  Chapter  2,  and  DELTA  is 
the  bias  factor. 

When  the  variance  of  the  queue  length  on  link  Lg  exceeds  a 
prespecified  threshold  value,  then  (5.3)  is  modified  as  follows: 

TK(D,Lg)  = Q(K,Lg)  x VAR(K,Lg)  + T (D)  + DELTA  (5.'*) 

where  VAR(K,Lg)  is  the  variance  of  the  queue  length  of  node  K on 
link  Lg.  The  term  Q(K,Lg)  x VAR(K,Lg)  represents  a bias  against 
unreliable  routes.  As  in  Chapter  2,  when  the  message  length  exceeds 
a certain  threshold,  the  routing  Is  not  done  through  the  delay 
tables  information  but  dynamically  via  the  following  formula 

L=min  [Q ( K , Lm)  +MVQ ( K , Lm) ] x 1 / 2+N*  ( K, D , Lm) xil -N*  ( K, D , Lm) x£  +T(D) 

Lm  avg 

(5.5) 

If  the  variance  of  the  queue  length  on  Lm  exceeds  a threshold 
equation  (5.5)  Is  modified  as  follows: 
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L®mln(Q(K,Lm)  x VAR(K,Lm)  + N (K,D,Lm)xA  - 
Lm 


- N(K,D,Lm)  x A + T(D)) 
' avg 


(5.6) 


When  a message  is  directed  to  another  subnet,  the  source  node 
determines  after  inspection  of  the  routing  tables  and  the  master  net 
delay  vectors,  the  local  central  node  which  minimizes  the  sum  of  the 
local  plus  master  net  delay  and  sends  the  message  to  It.  The 
equation  used  for  the  determination  of  the  minimum  delay  central  node 


T (D  , Lg  ) « min (min  T (j  ,Lg)  + MT  (S  )) 
K SD  Js  Lg  K Ss  jSs  D 

s 


(5.7) 


where  T (D  ,Lg  ) is  the  estimated  minimum  delay  to  go  from  source 
K SD 

node  K to  destination  node  D which  belongs  to  subnet  Sp,  through  the 

I 

outgoing  link  of  K,Lg'  . The  minimum  is  taken  over  the  central  nodes 
local  to  the  message  source  node.  Notice  that  the  local  central  nodes 
are  used  as  intermediate  destinations  to  get  to  the  destination  subnet 


When  messages  are  routed  outside  their  origination  subnet  and  their 
length  exceeds  a certain  threshold,  again  routing  is  done  dynamically 
via  the  following  formula: 

L ■ m!n{mln{[Q(K,Lm)  + MVQ(K,lm)  x 1/2  + N (K,j.  ,L  )^x  - 

]s  Lm 
s 

- N*(K,JS  ,Lm)Ax  ayg  + T(j$  )}  + MTj  (Sp)}  (5.8) 

s s S 

s 

Equation  (5.8)  is  the  analogous  of  (5.7).  If  the  variance  of  the 
queue  length  on  Lm  exceeds  a threshold  then  (5.8)  is  modified  as 
fol lows: 


(5.9) 


L = min{min{Q(K,Lm)  x VAR(K,Lm)  + N*(K,j  ,Lm)  xl  - 
j$  Lm  Ss 

s 

- N*(IC,J  (S0)  } 

s 

After  a message  arrives  at  a local  central  node.  It  is  handled 
by  the  masternet  routing  strategy. 


5.4  Master  Net  Routing  Strategy 

Ail  the  nodes  forming  the  master  net  are  central  nodes  of  the 
various  subnets.  Each  central  node  In  addition  to  the  local  routing 
tables  of  the  subnet  to  which  it  belongs.  Is  equipped  with  the  master 
net  routing  tables,  which  show  routes  and  minimal  delays  to  all  other 
central  nodes.  Using  the  master  net  delay  tables,  each  central  node 
computes  the  minimum  delay  vector,  as  In  equation  (5.2),  which  trans- 
mits to  the  nodes  of  the  subnet  to  which  it  belongs.  The  delay  table 
equation  for  the  master  net,  as  in  Chapter  2,  Is 


MT.(i,Lm)  *=  [Q(j  ,Lm)  + MVQ(j,Lm)]  x i/2  + MT(i)  + DELTA  (5.  i 0) 

where  MTj(l,Lm)  is  the  estimated  delay  to  go  from  node  j to  node  i 
through  link  lm.  MT(i)  is  the  delay  vector  local  to  the  master  net 
and  is  calculated  as  follows: 


MT(i)  ■ min  MT.(i,Lm) 


J 


(5.11) 


m 


If  the  variance  on  link  Lm  exceeds  a threshold,  then  (5.1 0)  becomes 


MTj (i ,Lm)  = Q(j ,Lm)  x VAR ( j , Lm)  + MT ( i)  + DELTA  (5.11) 


Speciai  routing  as  in  Chapter  2 is  done  via  the  formula: 
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ML  - mln{(MQ(j  ,Lm)  + MMVQ(j.Lm)]  x 1/2  + MN* (j , l ,Lm)xJl  - 
Lm 

- MN*(J,l,Lm)  x *avg  + MT(l)}  (5.12) 

where  M Is  added  In  every  symbol  to  indicate  that  the  master  net  is 
being  considered.  Similarly,  when  the  variance  on  Lm  exceeds  a 
threshold  (5.12)  becomes 

ML  » min{MQ(j  ,Lm)  x MVAR(J.Lm)  + MN*  (J  , l , Lm)  xH  - 
Lm 

- MN*(j  , i ,Lm)  x + MT  ( I > } (5.13) 

In  the  master  net  in  addition  to  the  routing  tables,  a connection 
vector  is  stored.  That  Is,  each  central  node  keeps  track  of  the  nodes 
of  Its  subnet,  that  it  can  reach  through  a local  route,  and  stores  the 
information  In  a N-dimensional  connection  vector.  The  1th  entry  of 
the  vector  Is  0 If  local  node  I is  unreachable  (because  it  Is  down  or 
disconnected);  It  Is  1 otherwise.  The  connect! non  vector  equation  is 

f I If  node  i is  reachable  from  central  node  j 

CVECTR.  (JO  - 

J ^ 0 otherwl se. 

Each  central  node,  therefore,  stores  the  connection  vectors,  one 
indicating  the  local  nodes  reachable  from  itself  and  the  rest  Indi- 
cating the  local  nodes  reachable  from  the  other  central  nodes  res- 
pectively. Fromthe  inspection  of  these  vectors,  each  central  node 
after  It  receives  a message  determines: 

(1)  the  message  can  be  directly  sent  to  a node  of  Its  subnet,  or 

(2)  must  be  routed  through  the  appropriate  central  node  to  reach 
its  destination,  or 
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(3)  cannot  be  delivered  because  t e destination  is  not  reachable 
from  any  central  node. 

The  update  of  the  delay  tables  Information  Is  done  as  described 
in  Chapter  2.  At  update,  the  subnets  update  their  local  routing 
tables.  The  master  net  updates  the  master  net  routing  tables.  It 
also  calculates  the  minimum  master  net  delay  vector  which  is  sent 
to  the  subnets  nodes  through  their  local  central  nodes. 

The  adaptive  routing  algorithm  for  large  computed  communication 
nets,  presented  above,  was  simulated. 

5.5  The  Simulation  Language 

The  simulation  language  used  to  fit  the  case  of  a large  network 
was  GASP1*. 

GASPlt,  developed  by  A.  Pritsker  at  Purdue  [PRIT  71*] , Is  an  event- 
oriented  language.  The  fundamental  approach  is  the  decomposition  of 
simulated  time  Into  events,  I.e.,  into  the  points  In  time  at  which  the 
state  of  the  system  could  change.  Each  event  routine  Is  a snapshot  of 
time.  Time  does  not  elapse  during  the  course  of  an  event  routine.  The 
event  routine  is  a detailed  description  of  the  decisions  and  steps  to 
be  executed  at  the  time  of  the  event.  Scheduling  an  event  simply  means 
filing  In  the  event  list  a record  identifying  the  event  type  and  the 
event  time.  Events  generally  either  initiate  an  activity  or  indicate 
the  end  of  an  activity.  In  coding  an  event,  one  looks  at  a point  in 
time  and  considers  ail  that  can  happen  at  that  point.  When  no  move 
can  take  place,  an  exit  from  the  routine  Is  made.  Simulated  time  is 
advanced  to  the  time  of  the  next  scheduled  event  and  the  procedure  Is 
repeated. 
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This  approach  Is  well  suited  to  situations  in  which  there  are 
a large  number  of  entitles  (In  this  case  messages)  flowing  through 
a series  of  relatively  simple  processes. 

5.5.1  Description  of  the  Simulation 

The  adaptive  algorithm  is  Implemented  in  terms  of  two  distinct 
events  in  time.  An  arrivai  event,  when  a message  arrives  In  the  system, 
and  an  end-of-service  event,  when  a message  leaves  a node  towards  the 
next  node  in  the  path  to  its  destination.  The  scheduling  of  the 
arrivai  event  is  done  by  using  exponential  distributed  interarrival 
time.  An  end-of-service  event  occurs  when  simulated  time  equal  to  the 
message  length  has  elapsed.  The  update  of  information  Is  also  associated 
with  an  event  which  is  scheduled  to  occur  after  the  occurance  of  a 
certain  number  of  message  arrival  events. 

The  activities  that  take  place  at  the  end  of  an  end-of-service 
event  are  keeping  data  of  the  queue  statistics  which  the  message  has 
recently  left.  When  a message  arrives  at  its  destination,  statistics 
are  kept  based  on  the  message  itself. 

The  GASP IV  output  is  statistics  based  on  message  observations  and 
statistics  based  on  link  queues  observations. 

5.5.2  Simulated  System  Characteristics 

The  network  simulated  is  shown  in  Fig.  The  partitioning 

algorithm  of  Chapter  b was  used  to  partition  the  network  In  three 
subnets.  Fig.  The  two  hierarchial  level  adaptive  routing  algorithm 
was  then  simulated  with  the  aid  of  the  computer.  The  two  level  net- 
work is  shown  in  Fig.  5.1.  The  master  net  Is  shown  separately  In 


Figure  5»  1 
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Fig.  5.1.  The  entire  network  consists  of  31  nodes  and  l^i  links.  Two 
nodes  are  selected  out  of  every  subnet  to  be  their  local  central  nodes. 
The  set  of  all  central  nodes  form  the  master  net.  All  links  are  two 
way.  The  node  connectivity  varies  from  3 to  6,  and  each  node  represents 
a switch. 

The  traffic  characteristics  assigned  to  the  network  are  as  follows: 

1.  Message  length:  Exponentially  distributed  with  a mean 
value  of  21,500  blts/mes,  or  mean  transmission  time  9 sec/mes 
since  the  link  capacity  Is  taken  to  be  C * 2^00  blts/sec. 

2.  Arrival  time:  Poisson  distributed  such  that  the  inter- 
arrlval  delay  Is  exponentially  distributed.  The  mean 
value  of  the  exponential  distribution  was  taken  equal  to 
1/1. 1*3  - .7  and  to  y - .5  for  two  traffic  loads  respec- 
tively. This  corresponds  to  1 . ^3  mes/sec  or  2 mes/sec 
arrival  rate. 

3.  Initial  and  destination  nodes:  Independently  drawn  from  a 
uniform  distribution  over  the  Integer  labels  of  the  nodes 
(1  to  31). 

The  simulation  was  run  several  times  for  varying  values  of: 
synchronous  update  period,  message  length  threshold  for  initiation 
of  asynchronous  update  (special  routing),  and  the  bias  factor  DELTA. 

The  update  information  messages,  that  the  nodes  send  to  each  other 
after  update  takes  place,  are  accounted  for  again  by  holding  every 
link  In  the  network  for  a short  period  of  time  (.01  sec,  in  this  case) 
after  update  calculations  are  over. 
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For  each  run,  the  -delay  for  each  message  was  measured  and  the 
average  delay  computed  per  message  was  computed  using  the  sample  mean 
formula. 


5.5.3  Conclus Ions 

Simulation  results  are  presented  here  for  two  different  traffic 
loads.  Two  update  periods,  UPDATE  20  ar.d  UPDATE  100,  were  considered. 
The  threshold  value  was  varied  from  a value  of  TRSLD  » 12  to  a value 
of  TRSLD  * 70.  The  Intermediate  values  v/ere  TRSLD  * 2C,  35,  50. 

TRSLD  ■ 12  would  correspond  to  the  highest  amount  of  special  routing 
allowed  and  TRSLD  * 70  to  the  lowest  or  practically  no  special  routing, 
since  very  few  message  lengths  will  exceed  a value  of  70. 

The  values  of  DELTA  were  taken  to  be  DELTA  * 10,  20,  50,  70. 

The  RATIO  of  the  ACTUAL  TOTAL  AVERAGE  DELAY  PER  MESSAGE  over  the 
IDEAL  TOTAL  DELAY  PER  MESSAGE  was  formed.  The  actual  total  average 
delay  per  message  Is  the  summation  of  the  transmission  plus  queueing 
delay.  The  ideal  delay  per  message  is  only  transmission  delay.  Data 
are  presented  for  a simulation  time  of  600  seconds. 

In  Fig.  5.2a,  the  RATIO  vs.  DELTA  was  plotted  for  two  update 
periods,  a low  traffic  of  1/y  ■ .7  sec/mes  and  a threshold  value  of  12. 

The  best  results  were  obtained  for  the  lowest  value  of  DELTA  used 
and  an  UPDATE  of  every  20  messages.  It  Is  of  Interest  that  when 
messages  are  strongly  bias  toward  the  direct  route  DELTA  50,70 
UPDATE  100  results  in  lower  delay  than  UPDATE  20.  The  delay  increases 
for  these  values  of  DELTA  because  the  direct  route  Is  overused  causing 
longer  queues  and  therefore  longer  delays.  In  addition  to  that, 
frequent  update  data  messages  create  further  delay  rather  than  improving 
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the  routing  decisions.  The  same  situation  is  shown  even  more  in 
Fig.  5.2b  where  the  traffic  is  higher,  i.e.,  i/y  ■ .5  sec/mes.. 

In  Fig.  5.3a  the  threshold  value  was  raised  to  TRSLD  ■ 35  and 
the  same  data  as  before  were  taken.  A higher  threshold  value  means 
that  special  routing  is  less  and  therefore  direct  routes  are  less 
encouraged  to  begin  with.  In  this  case  the  routing  depends  more  on 
the  update  of  routing  information.  Then  the  UPDATE  20  shows  less 
delay  than  the  UPDATE  100  curve.  In  higher  traffic  Fig.  5.3b  the 
same  situation  Is  true.  UPDATE  20  performs  considerably  Letter 
than  UPDATE  iOO. 

When  the  threshold  was  raised  even  more  to  a value  of  TRSLD  ■ 70 
frequent  update  (see  Fig.  5*^a)  performed  better  everywhere  for  iow 
traffic  (i/y  * .7  sec/mes).  In  higher  traffic  though  (i/y  ■ ,5mes/sec) 
frequent  update  did  not  always  show  less  delay.  The  UPDATE  iOO  (Fig. 

5.**b)  curve  is  fluctuating  considerably  as  a function  of  DELTA.  When 
the  routing  decisions  are  left  practically  to  the  updating  of  the 
routing  tables  (TRSLD  = 70)  and  the  traffic  is  high,  there  are 
instants  depending  on  traffic  peculiarities  where  practically  random 
routing  decisions  result  in  reduced  average  message  delay.  If  we 
define  the  output  rate  as  the  number  of  messages  delivered  during  the 
simulation  time  period,  then  the  output  rate  (Fig.  5.**b)  at  DELTA  = 50 
is  lower  for  UPDATE  100  (156  messages  delivered)  than  it  is  for  UPDATE  20. 
Therefore,  if  we  look  at  the  update  period  from  the  output  rate  point  of 
view,  UPDATE  20  still  performs  better  than  UPDATE  100  for  DELTA  = 20,50,70. 
For  DELTA  ■ 10  (Fig.  S.^b)  higher  delay  means  lower  output  rate  as  well 
UPDATE  20).  Therefore,  for  high  traffic,  alternate  routing  (DELTA  «*  10) 
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and  low  update  (UPDATE  ? 00)  gives  lower  delay  and  higher  output  rate. 

In  Fig.  5.5a  and  b the  effectiveness  of  the  special  routing  Is 
examined,  by  varying  the  TRSLD  value,  for  the  traffic  loads  respec- 
tively. The  update  Is  kept  constant  at  a value  of  20. 

The  special  routing  by  the  way  It  is  defined  always  bias  the 
routing  decisions  towards  the  direct  path.  The  bias  factor  DELTA 
also  bias  the  routing  decision  towards  the  direct  path,  by  an  amount 
directly  proportional  to  its  value.  Then  for  low  threshold  value,  l.e, , 
a large  number  of  messages  are  special  routed,  a low  value  of  DELTA 
shows  best  performance  for  both  traffic  loads.  That  is,  the  two  bias 
factors  (TRSLD,  DELlA)counteract  each  other.  When  the  low  TRSLD  value 
strongly  biases  most  routing  decisions  towards  the  direct  path,  then 
the  lowest  delay  Is  obtained  for  the  lowest  DELTA  value; that  is, the 
least  DELTA  bias  towards  the  direct  path.  When  the  threshold  value  Is 
high  (TRSLD  = 50,70)  higher  DELTA  values  result  in  minimum  delay  per 
message  (DELTA  = 50).  In  high  traffic  situations  (Fig.  5.5b),  the 
same  observations  are  more  evident.  This  Interaction  of  the  two  bias 
factors  is  expected  to  result  In  good  performance,  since  in  this  way 
direct  and  alternate  routes  are  used  In  a balanced  way  and  therefore  a 
minimum  message  delay  is  obtained. 

In  Fig.  5.6a  and  b the  same  data  are  plotted  when  the  update  is 
slower  UPDATE  100.  The  same  general  observations  are  true  and  in 
addition  the  slower  update  has  resulted  In  higher  message  delay  every- 
where. The  effectiveness  of  the  special  routing  Is  more  dominant 
(spreading  of  the  curves)  than  that  of  the  DELTA  bias  for  the  slower 
update  case.  (Compare  Fig.  5.5a,b,  Fig.  5.6a,b).  The  data  of  Fig.  5J&  b 


are  plotted  as  a function  of  the  threshold  value  In  Fig.  5.7.  A high 
value  of  OELTA  = 50  shows  minimum  delay  for  the  highest  TRSLD  ■ 70 
value.  For  low  TRSLD  = 12  the  DELTA  bias  factor  performance  is  about 
the  same  for  any  value  of  DELTA  which  Indicates  the  dominant  effect. 
In  this  case,  of  the  special  routing. 

As  It  is  defined  in  Chapter  2,  the  number  of  "excess  hops" 
measures  the  number  of  messages  delivered  which  followed 
other  than  the  direct  path  and  took  n excess  hops  to  arrive  at  their 
destination.  For  n = 10,  high  traffic,  and  slow  update,  the  Inter- 
action of  the  TRSLD  and  DELTA  are  examined  In  Fig.  5.8  in  relation 
to  the  excess  hops  measurement.  A low  value  of  DELTA  * 10  and  a high 
value  TRSLD  = 50  showed  least  looping.  Therefore,  a value  close  to 
the  average  message  length  would  prevent  looping  of  messages,  as  it 
was  claimed  in  the  algorithm,  in  conjunction  with  the  appropriate 
threshold  value. 

In  Fig.  5.  9 the  same  data  as  in  Fig.  5.8  are  plotted  against 
the  output  rate,  i.e.,  number  of  messages  delivered  to  their  des- 
tination. The  maximum  output  rate  occured  for  the  lowest  TRSLD  value 
and  lowest  DELTA  value.  The  same  case  also  shows  least  average  delay 
per  message  (see  Fig.  5.2b).  It  is  of  interest  to  note  that  the  same 
curve,  i.e.,  DELTA  = 10,  TRSLD  = 12  (see  Fig.  5.8)  gives  the  maximum 
number  of  messages  going  In  a loop.  This  is  because  a low  threshold 
value  will  route  even  short  length  messages  (TRSLD  = 12)  to 

direct  routes.  When  direct  routes  are  overused,  their  queues  are 
long  and  even  slight  variations  In  their  length  will  tend  to  make 
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the  short  messages  hop  from  queue  to  queue  without  actually  being 
delivered.  This  fact  creates  a lot  of  looping  for  a portion  of  short 
length  messages.  It  does  not  result  In  higher  average  delay  per  message 
because  a portion  of  short  length  message  delay  contributes  little  to  the 
average  message  delay.  Therefore,  looping  of  messages  Is  inefficient  but 
does  not  always  result  in  higher  average  message  delay. 

Summa  ry 

Best  performance  was  obtained  for  the  fastest  update  used,  a 
threshold  value  close  to  the  average  message  value  and  a bias  factor 
DELTA  approximately  equal  to  the  average  message  length. 

The  effect  of  the  update  was  as  expected,  i.e.,  in  general, 
fast  update  resulted  in  less  average  message  delay.  The  TRSLD 
variable  interacted  with  the  bias  DELTA  variable,  for  a balanced  use 
of  the  direct  route.  A low  DELTA  value,  biases  the  routing  decisions 
toward  alternating  routes  essentially,  while  a low  TRSLD  bias  routes 
most  messages  towards  the  direct  route.  With  this  combination  of  the 
two  bias  factors,  the  best  performance  was  obtained  everywhere.  The 
TRSLD  bias,  as  designed,  gave  a stronger  bias  towards  the  direct 
route  than  the  DELTA  bias.  A close  to  the  average  message  length  value 
of  DELTA  resulted  in  preventing  messages  of  going  Into  a loop.  The 
looping  of  messages  did  not  result  In  significant  average  message  delay 
under  the  best  performance  conditions  because  mostly  only  a portion  of 
short  length  messages  went  Into  loops  in  their  path  to  their  destinations. 
All  of  the  above  observations  were  true  for  both  low  and  high  message 
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Figure  5.3(a)  Ratio  w*.  Delta  for  TRSLD  - 35,  l/Y  * *7 
and  UP  - 20,  UP  - 100. 
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Tlqure  5.3(b)  Ratio  vt.  Delta  for  TRSLD  ■ 35,  1/Y  ■ .5 
and  ITT*  - 20,  UP  - 100. 
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Fioure  5.5(a)  Ratio  vs.  Delta  for  UP  = 20,  1/y  - .7  and 
TRSLD=1 2 , TRSLD-35,  TRSLD-50,  70. 
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FJoure  5.5(b)  Ratio  vs.  Delta  for  UP  - 20,  1/y  • •r- 

TRSLD-12,  TRSLD-20,  TRSLD-35,  TRSLD-50,70. 
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Figure  5.6(b)  Ratio  vs.  Delta  for  UP  - 100,  1/y  - .5  and 

TRSLD-12,  TRSLD-20,  TRSLD-35,  TRSLD-50,  TRSLD-70. 
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Figure  5.7  P.etlo  vs.  Threshold  for  UP  - 100,  1/y  - .5 
for  DELTA-10,  DELTA-20,  DELTA-50,  DELTA- 70. 
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Figure  5.9  Nun>ber  of  messages  delivered  vs.  Delta  for  UP  ■ 100, 
I /Y  - .5  and  DELTA  - 10,  DELTA  - 20,  DELTA  - 50, 
DELTA  - 70. 
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CHAPTER  6 

ROUTING  FOR  GEOMETRICAL  NETWORK  TOPOLOGIES 


I ntroduct Ion 

In  the  previous  chapters,  adaptive  routing  has  been  Investigated 
as  It  applies  to  distributed  communications  networks.  The  network's 
topology  was  not  exploited  In  the  routing  decisions.  In  general,  the 
algorithm  performs  very  well  in  a well  connected  network. 

When  the  network's  topology  follows  an  exact  geometrical  structure, 
then  this  fact  can  be  taken  into  consideration  In  the  routing  decisions. 

in  this  chapter  several  geometrical  network  topologies  are  discussed 
in  relation  to  the  routing  of  messages.  Specifically  lattice,  square, 
and  hexagonal  topologies  are  examined.  In  the  lattice  topology  case, 
routing  decisions  are  based  nn  current  information  about  the  network 
queues  and  a local  rule.  The  local  rule  is  a set  of  Instructions  that 
every  node  follows,  independent  of  the  network's  traffic  and  which 
takes  advantage  of  the  lattice  geometry.  When  a large  lattice  network 
is  considered,  then  the  application  of  the  local  rule  takes  care  of 
the  partitioning  of  the  network  also.  In  the  case  of  a square  or  an 
hexagonal  network  "response  routing"  depends  completely  on  a local 
rule  which  fits  specific  network  applications. 

The  discussion  In  this  chapter  intends  to  examine  how  the  network's 
topology  can  be  exploited  in  order  to  simplify  routing.  In  Section  6.1 
lattice  networks  are  Investigated.  In  Section  6.2,  square-x,  square-y, 
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and  hexagonal  networks  are  considered  along  with  possible  applications. 

6. 1 Routing  with  lattice  networks 

The  network  topology  Is  in  the  form  of  a lattice,  Fig.  6.1. 


Figure  6.1  A lattice  network  topology. 


A possible  partitioning  of  the  network  can  be  based  upon  the 
Shannon-Fano  method  [FULT  72].  The  network  Is  divided  Into  four  regions 
as  shown.  The  regions  are  numbered  by  a two  bit  address.  The  first  bit, 
either  1 or  0,  indicates  whether  the  region  is  above  or  below  the  origin, 
respectively.  Then  each  region  is  again  subdivided  into  four  parts  and 
each  subdivision  is  labeled  with  a two  bit  address  code.  Regions  are 
subdivided  by  this  technique  until  each  small  region  contains  one  node. 


A 
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For  the  example  shown  in  Fig.  6.1,  the  network  requires  three  levels  of 
partitioning.  Then  each  node  can  be  assigned  a unique  number  from  the 
part i t ions . 

For  example,  node  number  14  is  given  the  address  (11,  10,  01)  and 
node  number  33  is  given  the  address  (00,  11,  11).  For  this  type  of 
network,  the  node  addresses,  which  are  unique  by  the  partitioning  pro- 
cedure, provide  information  about  their  location  in  the  topological 
structure. 

Suppose  a message  is  to  be  routed  from  node  (11,  10,  01)  (node  33) 
to  node  (00,  11,  11)  (node  14).  By  comparing  the  first  pair  of  bits  in 
the  source  address  with  the  first  pair  of  bits  in  the  destination  node 
address,  it  can  be  determined  that  the  destination  node  is  located  above 
and  to  the  right  of  the  current  node.  A logical  choice  is  to  route  the 
message  to  the  right  (node  34)  or  up  (node  25)  since  both  of  these  nodes 
are  in  the  general  direction  of  the  destination  node.  When  the  message 
originating  at  node  33  arrives  at  any  node  (11,  xx,  xx)  contained  in  the 
major  region  of  destination,  there  is  a match  between  the  first  pair  of 
bits  in  the  address  codes  of  the  current  node  and  the  destination  node. 

At  this  point,  the  second  pair  of  bits  in  the  node  address  codes  is  used 
for  routing. 

When  more  tnan  one  path  out  of  any  node  exists  which  allows  a message 
to  proceed  in  the  general  direction  of  its  destination,  then  a choice  must 
be  made  as  to  which  line  should  be  used  to  exit  from  the  node.  Either  a 
selective  random  routing  procedure  could  be  used  or  the  shortest  outgoing 
queue.  In  this  way,  the  routing  information  depends  only  on  the  check 
on  the  addresses  of  the  nodes  and  a random  routine  or  a check  of  the 


shortest  queue. 


This  method  is  replacing  the  routing  decision  computations  by  a 
check  of  the  message  destination  address  plus  a check  of  the  shortest 
outgoing  queue  or  a random  decision  rule.  The  routing  Is  of  the 
alternate  type  but  under  restrictions,  since  once  a message  Is  In  a 
certain  address  level  It  cannot  return  to  the  level  It  came  from, 
even  if  in  the  meantime  there  are  available  routes  there.  This  fact 
helps  to  avoid  long  looping  of  messages.  The  disadvantage  of  this 
scheme  is  that  it  is  restricted  to  lattice  topology  networks.  It 
would  also  require  fast  transmission  communication  links  and  short 
packetized  messages  to  result  In  an  acceptable  performance.  It  is 
most  likely  to  be  used  |n  a network  when  immediate  response 

Is  not  important,  i.e.,  of  the  message  switched  type. 

6.2  Routing  with  Seicuk  networks 

The  network  consists  of  a large  array  of  nodes  each  having  limited 
logic  and  memory  [SAH!  7M . Each  node  is  connected  to  immediate  neighbors 
with  one-way  links  In  such  a way  that  a directed  path  exists  from  any  one 
node  to  any  other  node  (condition  of  connectivity).  By  varying  the  number 
of  links  per  node  and  the  arrangement  of  the  one-way  channels  (e.g.,  all 
incoming  links  adjacent  or  incoming  and  outgoing  links  alternating),  it  is 
possible  to  realize  many  networks.  Fig.  6.2  shows  different  node  types 
that  will  be  considered. 

In  Fig.  6.2(a)  and  (b)  the  node  types  result  In  square  networks.  Hence, 
they  will  be  called  square.  In  the  same  way,  Fig.  6.2(c)  and  (d)  the  node 
types  are  for  hexagonal  networks,  hence  the  term  hexagonal . Notice  that 
nodes  in  Fig.  6.2(a)  and  (c)  have  their  incoming  links  adjacent  to  each 
other.  In  Fig.  6.2(b)  and  (d)  the  Incoming  links  alternate  with  the  outgoing 


Figure 


6.2  Four  different  node  types  configurations. 


links.  Type  X will  refer  to  nodes  with  adjacent  incoming  links  and  t*pe_Y 
to  those  in  which  incoming  and  outgoing  links  alternate.  Thus,  the  node 
of  Fig.  6.2(a)  Is  square-X.  In  all  cases,  the  labeling  of  links  is  in  the 
clockwise  direction,  with  i,  denoting  the  first  incoming  link  and  i2  the 

second,  etc.,  and  similarly  the  outgoing  links. 

The  network  in  Fig.  6.3  contains  only  square-X  node,  the  arrangement 
of  which  is  such  that  it  is  possible  to  travel  in  straight  paths  from 
left  to  right,  right  to  left,  as  well  as  up  down  and  down  up. 


144 


When  all  links  are  one  way  and  the  condition  of  connectivity  Is 
satisfied,  the  network  is  called  unidj  rectional . 

In  such  kind  of  network  arrangements,  the  idea  Is:  Given  an  unidir- 
ectional net  be  able  to  find  a procedure  where  any  node  can  Interrogate 
the  entire  net  and  receive  responses  to  the  Interrogation  such  that 
routing  is  achieved  in  the  absence  of  any  knowledge  of  where  the  inter- 
rogator or  the  respondents  are  located  in  the  net. 

6.2.1  The  Selcuk  Proposition! 

Consider  the  square-X  net  of  Fig.  6.3.  Arbitrarily,  node  I Is  chosen 
to  be  the  source  node  that  will  initiate  the  interrogation.  The  quickest 
and  the  easiest  way  to  propagate  a message  to  the  net  is  to  have  each  node 
send  it  along  all  of  its  outgoing  links.  Since  the  net  is  connected, 
there  are  many  circular  paths  to  a given  node  and  that  will  result  in 
retransmissions  of  the  message  causing  it  to  remain  in  the  net  forever. 

The  echoes  can  be  stopped  by  requiring  that  a node  should  not  retransmit 
a particular  message  the  second  time  it  receives  it.  The  stopping  can  be 
achieved  either  through  a recognition  of  the  already  retransmitted  message 
or  by  having  a refractory  period  immediately  after  the  arrival  of  a message, 
during  which  further  messages  are  not  accepted. 

It  is  required  that  the  time  it  takes  for  a message  to  go  from  one 
node  to  the  next  be  the  same  at  all  nodes  and  define  this  interval  as 
unit  time.  Using  this  convention  and  the  stopping  of  second  arrivals, 

In  Fig.  6.4  the  propagation  of  a message  is  shown  initiated  at  node  I. 

The  heavy  elliptic  dots  show  the  input  channels  upon  which  the  message 
appears  for  the  first  time.  This  is  the  message  first  arrival  pattern. 


The  proposition  is  that  In  properly  constructed  nets  response 
routing  can  be  achieved  without  message  routing  Information  but  by  local 
applied  routing  rules,  in  the  nodes,  which  only  depend  on  the  first 
arrival  pattern  of  a message.  Therefore,  each  node  need  remember  only 
upon  which  of  its  input  channels  a given  message  first  arrived.  This 
proposition  is  called  the  Selcuk  principle.  Any  net  in  which  this  prin- 
ciple holds  will  be  identified  as  a Selcuk  network. 

Consider  the  net  in  Fig.  6.3  and  Fig.  6.4.  For  nodes  in  this  net, 
the  routing  rule  is  as  follows:  If  a message  arrives  on  link  ij,  when 
and  if  the  receiving  node  has  a response  to  this  message,  the  response 
is  routed  through  the  outgoing  link  0^;  if  It  arrives  on  1^  the 
routing  should  be  through  0^ ; and  If  it  arrives  on  lj  and  1 simul- 
taneously, the  routing  should  be  on  Oj.  This  rule  Is  clearly  local 
and  dependent  on  local  information. 


6.2.2  Propagation  Speed  in  Sguare-X  Selcuk  Nets 

It  is  shown  [SAHI  74]  that  in  a square-X  net  at  a unit  time  t the 
number  of  nodes  covered  Is 
r 2 at  t *»  1 


n 


11  at  t = 4 


l.  4(t-l)  at  t 2 2 t ^ 4 


The  total  number  of  nodes  N covered  in  general  Is 


N = E 4 ( t- 1 ) + 2-1  - 2 1 ( t—  1 ) +1  t i4 
1 

Similar  kinds  of  arrangements  (nets)  and  rules  can  be  achieved 
with  Hexagonal -Y  Selcuk  nets,  Square-Y  Selcuk  nets,  and  Hexagonal -X 


Selcut  nets. 


6.2.3  General  Remarks  and  Possible  Applications 

The  network  that  appears  In  Fig.  6.2  has  a geometric  structure. 

Such  a topology  Is  significant  only  to  the  extent  that  It  reflects 
the  assumption  of  equality  of  transmission  times.  As  long  as  the  time 
equality  is  satisfied,  for  example  for  compensating  delays,  the  net- 
work can  be  stretched  in  any  Imaginable  way  provided  that  the  connect- 
ivity patterns  remain  unchanged. 

The  Selcuk  networks  discussed  were  two  dimensional.  Sahin  claims 
that  cubic  networks  are  under  investigation  along  with  the  routing 
rules  they  should  satisfy. 

The  links  have  been  assumed  unidirectional.  It  is  proved  that 
if  bidirectional  links  are  to  be  used  instead  the  nodes  suffer  a 50 
percent  increase  in  logic  and  memory  with  only  negligible  improvement 
in  speed  of  service. 

Some  specific  applications  of  the  Selcuk  networks  are  proposed. 

As  an  example,  consider  a stock  market  which  is  distributed  such  that 
the  investors  can  fully  Interact  with  each  other.  In  such  a net,  each 
investor  would  be  interested  In  propagating  asking  price  and  quantity 
to  ali  other  investors,  i.e.,  send  a message  with  or  without  his 

identification  added  to  it.  The  investors  Interested  in  the  offer  or 
wishing  to  make  a different  bid  could  send  a response  with  their  iden- 
tification such  as  a telephone  or  code  number  attached  to  it. 

Another  example  of  computer  communication  that  would  benefit 
from  Selcuk  schemes  is  one  used  for  dynamic  resource  allocation  among 
many  branches  of  a company,  such  as  a car  rental  company.  Say  one 
branch  has  excess  cars.  It  could  then  send  a message  asking  for 


branches  with  certain  degrees  of  shortages.  The  responses  could  be 
required  to  contain  the  location  of  the  responding  branch.  Using 
mileage  indicated  by  the  located  code,  the  excess  cars  then  could 
optimally  be  allocated. 

An  area  under  extensive  investigation  Is  the  use  of  Selcuk 


networks  in  pattern  recognition  [SAHI  ] 


In  this  context,  the 


messages  are  viewed  as  initiators  of  local  processes.  Also  of 
interest  are  reported  findings  about  neuronel  networks.  Collectivity 
of  neurons  seems  to  constitute  networks  In  which  links  are  one  way. 
Such  nets  appear  to  support  spread  of  excitation.  There  is  also 
evidence  of  information  transfer  from  one  part  to  another  part  of 
the  network.  Efforts  to  find  a central  switchboard  to  facilitate 
such  transfers  have  failed.  While  there  is  no  claim  that  the  Selcuk 
principle  is  used  in  the  brain,  the  idea  behind  this  principle  could 
be  a useful  starting  point  for  a new  approach  to  understanding 
brain  behavior  [SAHI  733 • 
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Appendix  I 


I.  The  AUTODIN  System  (from  [Hous  75] ) 

AUTOOIN  [AUTO  653  Is  a message  switched  store-and-forward  data 
communication  system  which  may  be  considered  as  two  sections:  CONUS 
AUTODIN  in  the  continental  United  States,  which  is  managed  by  a private 
contractor,  and  overseas  AUTODIN,  which  is  managed  by  the  Department 
of  Defense.  The  characteristics  of  the  two  systems  are  different,  and 
this  study  was  primarily  concerned  with  overseas  AUTODIN.  AUTODIN 
works  on  the  store-and-forward  principle,  in  which  the  entire  message 
is  accumulated  at  the  AUTODIN  switch  and  receipt  Is  verified  before 
initiating  further  transmission.  This  represents  a fundamental  dif- 
ference from  a packet ized  system  such  as  the  ARPANET  [ROBE  70]  or  the 
projected  AUTODIN  II,  in  which  the  message  is  broken  up  into  "packets" 
of  uniform  size,  each  packet  is  treated  as  an  independent  message,  and 
the  packets  are  reassembled  at  the  destination.  In  AUTODIN,  the  message 
length  can  vary  from  i kb  to  336  kb  with  an  average  of  2 i . 5 kb. 

The  second  feature  of  overseas  AUTODIN,  a feature  Important  in 
any  control  scheme,  is  the  relatively  slow  speed  of  the  overseas 
interswitch  Jinks,  which  transmit  data  at  2. Allbits.  This  means  that. 

In  general,  the  limiting  factor  In  system  performance  is  the  inter- 
switch deiay,  and  the  system  control  algorithm  will  be  concerned  with 
control  of  interswitch  routing. 
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The  low  transmission  rate  means  that  the  longest  allowable  message 
can  take  13  minutes  to  transmit,  and  thus  the  long  messages  have  a dis- 
proportionate effect  on  the  delay  of  the  other  messages.  The  present 
routing  scheme  Involves  "lookup"  tables  stored  In  every  switch.  These 
tables  have  as  many  entries  as  there  are  switches,  each  entry  labelled 

with  a destination  switch,  and  the  entry  gives  the  outgoing  link  on 
which  messages  to  that  destination  switch  are  sent.  At  present,  these 

tables  are  updated  by  a central  control,  but  usually  at  long  intervals 
and  after  computer  simulation  of  the  change.  The  system  Is  unable  to 
respond  quickly  to  sudden  variations  in  loading  or  to  equipment  failures. 
In  addition,  the  system  Is  deliberately  underdesigned  for  the  sake  of 
economy,  and  thus  peaks  In  loads  can  cause  larger  delays.  The  short 
delay  required  for  Important  messages  Is  ensured  by  a priority  (pre- 
cedence) system.  These  priorities  are  assigned  to  messages  by  sub- 
scribers, although  not  all  subscribers  can  assign  the  highest-level 
priori  ties. 
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APPENDIX  II 


111.  The  FO  Algorithm 

In  section  3.4.1  two  conditions  were  given  for  the  convergence 


of  the  FD  method.  In  (I)  AP(f)  > 0 where  AP(f)  - P(f)  - P(FD®f).  Then 


M 


AP(f)  - P(f)  - P(FD  f)  SAX  E £K(fc’VK) 

K”  1 


A 3P 

where  £K  “ and  v shortest  route  flow  under  matric  £^.  Therefore, 


by  condition  (i) 


M 


E i 0 v £ Fa 


K'  K K 


K*=l 

and  by  condition  (11) 


H 


E £ (f  -v„)  < e -*■  f stationary 
K=1  K K K 


where  e positive  tolerance. 


Phase  1 


1.  With  p » 1 , let  f be  the  shortest  route  flow  computed  at  f ■ 0, 

u 


l.e.v  with  metric  = [3T/3f^]^  bq.  Let  n ■ 0. 

f n K 

K 

2.  Let  a «•  max  ( 7 — ). 
n K CK 


If  o /p  <1,  let  f^  ■ fn/P  and  go  to  Phase  2.  Otherwise,  let 
n n » n 


p j ■=•  pn(l  - e (l-an))/cn,  where  e Is  a proper  tolerance,  0 < e < 1 


Let  gn+1  = fn(pMj.,/pR).  Go  to  3. 


n+r  n 


158 


3.  Let  fMl  - FW»MI 

^ ««r 

4.  If  n ■ 0,  go  to  6 

M n+1 

5.  If  | S ^k^vK  " ®K  H<  6 and  l^n+l  ” %l  <^’  w^ere  ® and  ^ 

K®  I 

are  proper  positive  tolerances,  and  v Is  the  shortest  route 
flow  computed  at  gn+* , stop:  the  problem  Is  Infeasible  within 
tolerances  8 and  <S.  Otherwise,  go  to  6. 

6.  Let  n ■ n+1  and  go  to  2. 

Phase  2 

1 . Let  n • 0 

2.  fn+1  - FD®f" 

<s* 

H n 

3.  If  | I (f^  - vK)|<  0 , where  0 Is  a proper  positive  tolerance, 

K-l 

stop:  f Is  optimal  within  a tolerance  0.  Otherwise,  let 
n - n+1  and  go  to  2. 

In  every  Iteration  of  the  algorithm  there  is  a shortest  route 

computation.  Therefore,  we  have  both  the  flow  assignment  in  the  links 

and  the  shortest  route  routing.  At  the  last  Iteration  then  we  obtain  the 

optimal  flow  assignment  along  with  the  optimal  routing  which  Is  the 

shortest  route  for  the  optimal  flow  vector. 

Note  that  In  the  first  step  of  Phase  l a shortest  route 

computation  Is  done  at  f * 0,  and  f®  Is  obtained.  With  f^  *+  0, 

there  are  no  queues  In  the  network.  Then  the  shortest  route 

minimizes  a function  of  the  sum  of  transmission  plus  propagation 

0 

delay.  Note  also  that  f computed  In  this  way  guarantees  that  f eFa 
since  a shortest  route  flow  Is  an  extreme  point  eFa. 
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112.  Shortest  Route  Algorithm 

There  are  many  shortest  route  algorithms  available  In  the 
literature  [FORD  62],  [FRiS  71],  [FLOY  62],  The  one  used  here  Is 
presented  by  Frlsh  [FRIS  71]. 

Let  4 be  the  weight  of  link  K,  and  let  K be  the  link  connecting 
node  1 to  node  j.  Then  4^  can  be  written  as  4^  jj. 

1.  Assign  all  nodes  1 labels  of  the  form  (♦,6(0)  where  6(s)  = 0 
and  6(a)  = 00  for  a i1  S, 

2.  Find  a link  (!,j)  such  that  6(1)  + 4^.  jj  < 6(j).  If  such  a link 
is  found,  change  the  label  on  node  j to  (1,6(1)  + 4^  jj)*  Repeat  this 
operation  until  no  such  branch  is  found. 

3.  To  Identify  the  nodes  in  the  shortest  directed  s - a path  a s: 

a)  Let  l *■  a. 

b)  identify  node  K from  the  label  (K, 6 ( I ) ) on  vertex  1.  If  K 
does  not  exist,  then  there  is  no  s - a path  In  the  graph. 

c)  Let  i = K.  if  1 = S,  terminate.  Otherwise  return  to  (b) . 

It  is  proved  [FRIS  71]  that  the  shortest  path  algorithm  terminates  In 
a finite  number  of  steps  and  finds  the  shortest  s - a path,  provided 
that  the  sum  of  the  link  weights  along  a path  Is  positive. 
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APPENDIX  III 


Frisch's  connectivity  labeling  algorithm  is  used  to  find  a feasible 
flow  path,  if  any,  between  a single  source  and  a single  destination  in 
each  iteration  of  the  labeling  process.  This  algorithm  Is  generalized 
to  find  more  than  one  feasible  flow  path,  If  multiple  flow  paths  exist, 
between  a set  of  sources  and  a set  of  destinations  In  each 
iteration  of  the  labeling  process. 

Let  a network  be  denoted  as  G ■ (Vf  ),  and  let  a set  of  sources  be 
S =■  (S|,...,s  ) and  a set  of  destinations  T « (tj,...,t  ),  where  p and  q 
are  two  positive  integers.  A generalized  version  of  Frisch’s  labeling 
algorithm  is  given  below.  The  algorithm  assumes  a zero  initial  flow. 


Routine  A — Labeling  Process 


1.  Label  s.,...,s  by  (s, ,+),..., (s  , +) . Now  S «*  (s.,...,s  ) is 

l p l p l p 

+ iabeied  and  + unscanned.  Ail  other  nodes  are  unlabeled  and  unscanned. 


2.  For  any  node  x e T which  is  labeled  and  unscanned: 

(a)  Label  every  node  y as  (x,  -)  if  (y,x)  eF  , y is  - unlabeled, 
f (y ,x)  - i , yeS  and 

(i)  x is  + labeled,  + unscanned  and  - unlabeled,  or 
(ii)  x is  - labeled  and  - unscanned. 

If  in  (ii)  x Is  + labeled  and  + scanned,  then  add  "l"  to  the  label  on  y. 
If  y is  + labeled  and  + unscanned,  then  erase  the  + label. 

(b)  Label  every  node  y as  (k,  +)  If  (x,y)  e P,  y is  unlabeled,  (or 
y is  + labeled  and  in  T) , f(x,y)  ■ 0,  y e S and 

(1)  xfS  and  x Is  + labeled  and  + unscanned,  or 
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(II)  x eS,  x Is  + labeled,  + unscanned,  and  - unlabeled  and 
f(x,V)  ■*  0,  or 

(ill)  x cS  and  x Is  - labeled  and  - unscanned. 

If  in  (iil)  x Is  + labeled  and  + scanned,  then  add  "I"  to  the  label  on  y. 

(c)  In  cases  a(l),  b ( 1 ) and  (li),  + scan  x.  In  cases  a ( I i ) and 
b ( I i i ) , - scan  x. 

3.  Repeat  step  2 until  either  no  more  labels  can  be  assigned  and  no 
t eT  is  labeled  or  some  t eT  Is  labeled. 

(a)  In  the  former  case,  terminate  the  procedure. 

(b)  In  the  latter  case,  use  routine  B to  obtain  a feasible  flow 
path,  if  possible.  Then,  If  any  further  labeling  Is  possible,  go  to 
step  2;  otherwise,  go  to  step  I. 

Routine  B --  Flow  Change 

1.  Let  z ■ t and  go  to  step  2. 

2.  (a)  If  the  label  on  z is  (w,  + and 

(I)  f(w,z)  " 0,  Increase  f(w,z)  by  I and  go  to  step  3;  or 
(ii)  f(w,z)  * 1,  go  to  step  5. 

(b)  If  the  label  on  z Is  (w,  -)  and 

(I)  f(z,w)  “ I,  decrease  f(z,w)  by  I and  go  to  step  3;  or 
(II)  f(z,w)  * 0,  go  to  step  5. 

3.  If  w eS,  then  return  to  step  3(ii)  of  Routine  A.  Otherwise,  go 
to  step  4. 

4.  If  in  step  2 the  label  (w,  +)  or  (w,  -)  Is  followed  by  an  "I," 
then  replace  the  current  value  of  z by  w and  go  to  step  2(b).  If  In 
step  2 the  label  is  not  followed  by  an  "!,"  then  replace  z by  w and  go 
to  step  2. 


5.  No  feasible  flow  path  Is  possible.  Restore  the  old  flow 
pattern  existing  before  the  application  of  Routine  B,  and  go  to  step 
3(||)  of  Routine  A. 
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5 MISSION  b 

\ of  l 

ft  Rome  Air  Development  Center 


RADC  plans  and  conducts  research,  exploratory  and  advanced 
development  programs  in  command,  control,  and  coamuni cations 
(C3)  activities,  and  in  the  C3  areas  of  information  sciences 
and  intelligence.  The  principal  technical  mission  areas 
are  communications,  electromagnetic  guidance  and  control, 
surveillance  of  ground  and  aerospace  objects.  Intelligence 
data  collection  and  handling,  information  system  technology, 
ionospheric  propagation,  solid  state  sciences,  micromave 
physics  and  electronic  reliability,  maintainability  and 
compatibility . 


